fd -t f -0 | xargs -0 -I {} sh -c 'echo "File: {}"; cat {}'
Friday, October 18, 2024
Find content of all relevant files in a project respecting .gitignore
Tuesday, August 06, 2024
Remove server header in IIS to stop disclosing running version(s)
Set attribute removeServerHeader="true" in web.config node configuration."system.webServer".security.requestFiltering.
Thursday, July 18, 2024
When the clock on MAC OSX goes off
To sync your macOS clock and prevent it from drifting after sleep cycles, follow these steps:
- Run this command to sync time:
- Remove the current time settings:
- After reboot, turn on "Set time and date automatically" in System Settings > Date & Time. This should ensure your clock remains accurate.
- Run this command to sync time:
sudo sntp -sS time.apple.com- Go to System Settings > Date & Time and turn off "Set time and date automatically."
- Remove the current time settings:
sudo rm /var/db/timed/com.apple.timed.plist- Reboot the Mac.
- After reboot, turn on "Set time and date automatically" in System Settings > Date & Time. This should ensure your clock remains accurate.
Monday, May 27, 2024
Sunday, April 14, 2024
Minimal kubernetes pods log shipping with fluent-bit
I just released how to ship kubernetes cluster pod logs to Amazon Simple Storage Service (AWS S3) using fluent-bit. Platform engineers should do their best to keep cloud independence. This setup will work for Azure AKS, Google GKE and Amazon EKS. The choice for S3 is just a pick and not a preference. Google Cloud Storage (GCS) or Azure Blob Storage (ABS) will do the job as well (just check fluent-bit docs and adapt the code below for your choice).
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# This setup works seemlessly if you deploy microservices in any of the big 3 cloud providers using terraform and helm as I described in the following github projects: | |
# https://github.com/nestoru/gcp-microservices-poc | |
# https://github.com/nestoru/azure-microservices-poc | |
# https://github.com/nestoru/aws-microservices-poc | |
#1. Set in your protected environment specific IaC project the helm vars: | |
# api-helm/values.yaml | |
fluentBit: | |
enabled: true | |
s3: | |
accessKey: *** | |
secretKey: *** | |
bucket: *** | |
region: *** | |
#2. Define a deployment descriptor for a fluent-bit pod to be deployed by helm | |
# api-helm/templates/fluent-bit-deployment.yaml | |
{{- if .Values.fluentBit.enabled }} | |
apiVersion: apps/v1 | |
kind: Deployment | |
metadata: | |
name: fluent-bit | |
labels: | |
app: fluent-bit | |
spec: | |
replicas: 1 | |
selector: | |
matchLabels: | |
app: fluent-bit | |
template: | |
metadata: | |
labels: | |
app: fluent-bit | |
spec: | |
serviceAccountName: fluent-bit | |
containers: | |
- name: fluent-bit | |
image: fluent/fluent-bit:latest | |
volumeMounts: | |
- name: fluent-bit-config | |
mountPath: /fluent-bit/etc/ | |
- name: varlog | |
mountPath: /var/log | |
readOnly: false # Ensure this is false to allow writing | |
env: | |
- name: AWS_ACCESS_KEY_ID | |
valueFrom: | |
secretKeyRef: | |
name: fluent-bit-aws-credentials | |
key: AWS_ACCESS_KEY_ID | |
- name: AWS_SECRET_ACCESS_KEY | |
valueFrom: | |
secretKeyRef: | |
name: fluent-bit-aws-credentials | |
key: AWS_SECRET_ACCESS_KEY | |
volumes: | |
- name: fluent-bit-config | |
configMap: | |
name: fluent-bit-config | |
- name: varlog | |
hostPath: | |
path: /var/log | |
type: DirectoryOrCreate # This ensures the directory is created if it doesn't exist | |
{{- end }} | |
#3. Define a configmap kubernetes descriptor containing your settings for log collection for helm to deploy | |
# api-helm/templates/fluent-bit-configmap.yaml | |
{{- if .Values.fluentBit.enabled }} | |
apiVersion: v1 | |
kind: ConfigMap | |
metadata: | |
name: fluent-bit-config | |
data: | |
fluent-bit.conf: |- | |
[SERVICE] | |
Flush 5 | |
Log_Level info | |
Parsers_File parsers.conf | |
HTTP_Server On | |
HTTP_Listen 0.0.0.0 | |
HTTP_Port 2020 | |
[INPUT] | |
Name tail | |
Path /var/log/containers/{{ .Values.appServiceName }}-{{ .Values.majorVersion }}-test*.log | |
Tag {{ .Values.appServiceName }}.{{ .Values.majorVersion }} | |
Parser docker | |
DB /var/log/flb_kube.db | |
Mem_Buf_Limit 5MB | |
Skip_Long_Lines On | |
[OUTPUT] | |
Name s3 | |
Match {{ .Values.appServiceName }}.{{ .Values.majorVersion }} | |
bucket {{ .Values.fluentBit.s3.bucket }} | |
region {{ .Values.fluentBit.s3.region }} | |
s3_key_format /%Y/%m/%d/{{ .Values.appServiceName }}-{{ .Values.majorVersion }}.log | |
upload_timeout 1m | |
use_put_object On | |
storage.total_limit_size 1G | |
parsers.conf: |- | |
[PARSER] | |
Name docker | |
Format json | |
Time_Key time | |
Time_Format %Y-%m-%dT%H:%M:%S.%L | |
Time_Keep On | |
{{- end }} | |
#4. Define a kubernetes secret descriptor for the S3 bucket to be deployed by helm | |
# api-helm/templates/fluent-bit-secret.yaml | |
apiVersion: v1 | |
kind: Secret | |
metadata: | |
name: fluent-bit-aws-credentials | |
type: Opaque | |
stringData: # Note: Using `stringData` for convenience with plain text values | |
AWS_ACCESS_KEY_ID: {{ .Values.fluentBit.s3.accessKey }} | |
AWS_SECRET_ACCESS_KEY: {{ .Values.fluentBit.s3.secretKey }} | |
#5 Define a kubernetes serviceaccount descriptor to be deployed by helm | |
# api-helm/templates/fluent-bit-service-account.yaml | |
apiVersion: v1 | |
kind: ServiceAccount | |
metadata: | |
name: fluent-bit | |
namespace: {{ .Release.Namespace | default "default" }} |
Wednesday, March 20, 2024
Print the content of all zipped files with names containing a regex
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# | |
# The below will print the content of all zipped files with names containing the keyword "application" | |
# ./print_zip_files_content.sh app.jar '.*application.*' | |
# | |
if [ "$#" -ne 2 ]; then | |
echo "Usage: $0 <zip file path> <regex containing filename>" | |
exit 1 | |
fi | |
ZIP_FILE="$1" | |
REGEX="$2" | |
# Using unzip -l to list files, then grep to filter by regex. awk to print the 4th column to the end. | |
# This handles spaces in filenames more gracefully. | |
unzip -l "$ZIP_FILE" | grep -P "$REGEX" | awk '{print $4}' | while read -r file; do | |
if [[ ! -z "$file" ]]; then | |
echo "File $file:" | |
unzip -p "$ZIP_FILE" "$file" | |
echo -e "\n" | |
fi | |
done |
Sunday, March 10, 2024
Securely sharing the content of your MAC OSX ~/.zshrc and other profile configuration files
cat ~/.zshrc | sed -E "s/(export[^=]*=).*/\1***/"
Friday, February 23, 2024
Tuesday, February 20, 2024
Run tree command respecting .gitignore
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# /opt/scripts/tree-gitignore.sh | |
# Author: Nestor Urquiza | |
# Date: 20240220 | |
# Description: A tree wrapper to show contents of a git project respecting .gitignore | |
# Usage: cd git-project && /opt/scripts/tree-gitignore.sh | |
cmd="tree -a -I '.git'" | |
# Read each line from .gitignore | |
while IFS= read -r line; do | |
# Skip empty lines and comments | |
if [[ "$line" != "" && "$line" != \#* ]]; then | |
# Remove leading "**/" from patterns, if present | |
pattern="${line/\*\*\//}" | |
# Remove trailing "/*" from patterns, if present | |
pattern="${pattern/\/\*/}" | |
# Append each cleaned pattern as an ignore option | |
cmd+=" -I '$pattern'" | |
fi | |
done < .gitignore | |
# Execute the constructed command | |
eval $cmd |
Monday, February 19, 2024
My .vimrc for google (and amazon) cloud shell
Will keep here what I am using in google (and amazon) cloud shells.
" keep all defaults source $VIMRUNTIME/defaults.vim " disable visual mode when using mouse set mouse=c " keep all defaults source $VIMRUNTIME/defaults.vim " disable visual mode when using mouse set mouse=c " paste as copied set paste " indentation set tabstop=2 " Set the width of a tab character to 2 spaces set shiftwidth=2 " Set the number of spaces to use for each step of (auto)indent set expandtab " Convert tabs to spaces set softtabstop=2 " Set the number of spaces a Tab counts for while performing editing operations, like inserting a Tab or using BS set smartindent " Enable smart indenting for new lines
Sharing code without using git
Sometimes you do not want to commit to git for security reasons and yet you want to test your code somewhere else.
Run the below from your local copy of the git repository:
Run the below from your local copy of the git repository:
rsync -av --exclude-from='.gitignore' --exclude='.git' . ~/Downloads/my-repo-copy cd ~/Downloads/my-repo-copy zip -r ~/Downloads/my-repo-copy.zip ./Copy the content to the remote location and run the below:
mkdir my-repo mv my-repo-copy.zip my-repo/ cd my-repo unzip my-repo-copy.zip rm my-repo-copy.zip
Subscribe to:
Posts (Atom)