Tuesday, August 06, 2024

Remove server header in IIS to stop disclosing running version(s)

Set attribute removeServerHeader="true" in web.config node configuration."system.webServer".security.requestFiltering.

Thursday, July 18, 2024

When the clock on MAC OSX goes off

To sync your macOS clock and prevent it from drifting after sleep cycles, follow these steps:

- Run this command to sync time:
sudo sntp -sS time.apple.com
- Go to System Settings > Date & Time and turn off "Set time and date automatically."

- Remove the current time settings:
sudo rm /var/db/timed/com.apple.timed.plist
- Reboot the Mac.

- After reboot, turn on "Set time and date automatically" in System Settings > Date & Time. This should ensure your clock remains accurate.

Sunday, April 14, 2024

Minimal kubernetes pods log shipping with fluent-bit

I just released how to ship kubernetes cluster pod logs to Amazon Simple Storage Service (AWS S3) using fluent-bit. Platform engineers should do their best to keep cloud independence. This setup will work for Azure AKS, Google GKE and Amazon EKS. The choice for S3 is just a pick and not a preference. Google Cloud Storage (GCS) or Azure Blob Storage (ABS) will do the job as well (just check fluent-bit docs and adapt the code below for your choice).
# This setup works seemlessly if you deploy microservices in any of the big 3 cloud providers using terraform and helm as I described in the following github projects:
# https://github.com/nestoru/gcp-microservices-poc
# https://github.com/nestoru/azure-microservices-poc
# https://github.com/nestoru/aws-microservices-poc
#1. Set in your protected environment specific IaC project the helm vars:
# api-helm/values.yaml
fluentBit:
enabled: true
s3:
accessKey: ***
secretKey: ***
bucket: ***
region: ***
#2. Define a deployment descriptor for a fluent-bit pod to be deployed by helm
# api-helm/templates/fluent-bit-deployment.yaml
{{- if .Values.fluentBit.enabled }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: fluent-bit
labels:
app: fluent-bit
spec:
replicas: 1
selector:
matchLabels:
app: fluent-bit
template:
metadata:
labels:
app: fluent-bit
spec:
serviceAccountName: fluent-bit
containers:
- name: fluent-bit
image: fluent/fluent-bit:latest
volumeMounts:
- name: fluent-bit-config
mountPath: /fluent-bit/etc/
- name: varlog
mountPath: /var/log
readOnly: false # Ensure this is false to allow writing
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: fluent-bit-aws-credentials
key: AWS_ACCESS_KEY_ID
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: fluent-bit-aws-credentials
key: AWS_SECRET_ACCESS_KEY
volumes:
- name: fluent-bit-config
configMap:
name: fluent-bit-config
- name: varlog
hostPath:
path: /var/log
type: DirectoryOrCreate # This ensures the directory is created if it doesn't exist
{{- end }}
#3. Define a configmap kubernetes descriptor containing your settings for log collection for helm to deploy
# api-helm/templates/fluent-bit-configmap.yaml
{{- if .Values.fluentBit.enabled }}
apiVersion: v1
kind: ConfigMap
metadata:
name: fluent-bit-config
data:
fluent-bit.conf: |-
[SERVICE]
Flush 5
Log_Level info
Parsers_File parsers.conf
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_Port 2020
[INPUT]
Name tail
Path /var/log/containers/{{ .Values.appServiceName }}-{{ .Values.majorVersion }}-test*.log
Tag {{ .Values.appServiceName }}.{{ .Values.majorVersion }}
Parser docker
DB /var/log/flb_kube.db
Mem_Buf_Limit 5MB
Skip_Long_Lines On
[OUTPUT]
Name s3
Match {{ .Values.appServiceName }}.{{ .Values.majorVersion }}
bucket {{ .Values.fluentBit.s3.bucket }}
region {{ .Values.fluentBit.s3.region }}
s3_key_format /%Y/%m/%d/{{ .Values.appServiceName }}-{{ .Values.majorVersion }}.log
upload_timeout 1m
use_put_object On
storage.total_limit_size 1G
parsers.conf: |-
[PARSER]
Name docker
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
{{- end }}
#4. Define a kubernetes secret descriptor for the S3 bucket to be deployed by helm
# api-helm/templates/fluent-bit-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: fluent-bit-aws-credentials
type: Opaque
stringData: # Note: Using `stringData` for convenience with plain text values
AWS_ACCESS_KEY_ID: {{ .Values.fluentBit.s3.accessKey }}
AWS_SECRET_ACCESS_KEY: {{ .Values.fluentBit.s3.secretKey }}
#5 Define a kubernetes serviceaccount descriptor to be deployed by helm
# api-helm/templates/fluent-bit-service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: fluent-bit
namespace: {{ .Release.Namespace | default "default" }}

Wednesday, March 20, 2024

Print the content of all zipped files with names containing a regex

#!/bin/bash
#
# The below will print the content of all zipped files with names containing the keyword "application"
# ./print_zip_files_content.sh app.jar '.*application.*'
#
if [ "$#" -ne 2 ]; then
echo "Usage: $0 <zip file path> <regex containing filename>"
exit 1
fi
ZIP_FILE="$1"
REGEX="$2"
# Using unzip -l to list files, then grep to filter by regex. awk to print the 4th column to the end.
# This handles spaces in filenames more gracefully.
unzip -l "$ZIP_FILE" | grep -P "$REGEX" | awk '{print $4}' | while read -r file; do
if [[ ! -z "$file" ]]; then
echo "File $file:"
unzip -p "$ZIP_FILE" "$file"
echo -e "\n"
fi
done

Tuesday, February 20, 2024

Run tree command respecting .gitignore

#!/bin/bash
# /opt/scripts/tree-gitignore.sh
# Author: Nestor Urquiza
# Date: 20240220
# Description: A tree wrapper to show contents of a git project respecting .gitignore
# Usage: cd git-project && /opt/scripts/tree-gitignore.sh
cmd="tree -a -I '.git'"
# Read each line from .gitignore
while IFS= read -r line; do
# Skip empty lines and comments
if [[ "$line" != "" && "$line" != \#* ]]; then
# Remove leading "**/" from patterns, if present
pattern="${line/\*\*\//}"
# Remove trailing "/*" from patterns, if present
pattern="${pattern/\/\*/}"
# Append each cleaned pattern as an ignore option
cmd+=" -I '$pattern'"
fi
done < .gitignore
# Execute the constructed command
eval $cmd

Monday, February 19, 2024

My .vimrc for google (and amazon) cloud shell

Will keep here what I am using in google (and amazon) cloud shells.
" keep all defaults
source $VIMRUNTIME/defaults.vim
" disable visual mode when using mouse
set mouse=c
" keep all defaults
source $VIMRUNTIME/defaults.vim
" disable visual mode when using mouse
set mouse=c
" paste as copied
set paste
" indentation
set tabstop=2        " Set the width of a tab character to 2 spaces
set shiftwidth=2     " Set the number of spaces to use for each step of (auto)indent
set expandtab        " Convert tabs to spaces
set softtabstop=2    " Set the number of spaces a Tab counts for while performing editing operations, like inserting a Tab or using BS
set smartindent      " Enable smart indenting for new lines

Sharing code without using git

Sometimes you do not want to commit to git for security reasons and yet you want to test your code somewhere else.
Run the below from your local copy of the git repository:
rsync -av --exclude-from='.gitignore' --exclude='.git' . ~/Downloads/my-repo-copy
cd ~/Downloads/my-repo-copy
zip -r ~/Downloads/my-repo-copy.zip ./
Copy the content to the remote location and run the below:
mkdir my-repo
mv my-repo-copy.zip my-repo/
cd my-repo
unzip my-repo-copy.zip
rm my-repo-copy.zip

Followers