Tuesday, August 06, 2024

Remove server header in IIS to stop disclosing running version(s)

Set attribute removeServerHeader="true" in web.config node configuration."system.webServer".security.requestFiltering.

Thursday, July 18, 2024

When the clock on MAC OSX goes off

To sync your macOS clock and prevent it from drifting after sleep cycles, follow these steps:

- Run this command to sync time:
sudo sntp -sS time.apple.com
- Go to System Settings > Date & Time and turn off "Set time and date automatically."

- Remove the current time settings:
sudo rm /var/db/timed/com.apple.timed.plist
- Reboot the Mac.

- After reboot, turn on "Set time and date automatically" in System Settings > Date & Time. This should ensure your clock remains accurate.

Sunday, April 14, 2024

Minimal kubernetes pods log shipping with fluent-bit

I just released how to ship kubernetes cluster pod logs to Amazon Simple Storage Service (AWS S3) using fluent-bit. Platform engineers should do their best to keep cloud independence. This setup will work for Azure AKS, Google GKE and Amazon EKS. The choice for S3 is just a pick and not a preference. Google Cloud Storage (GCS) or Azure Blob Storage (ABS) will do the job as well (just check fluent-bit docs and adapt the code below for your choice).
# This setup works seemlessly if you deploy microservices in any of the big 3 cloud providers using terraform and helm as I described in the following github projects:
# https://github.com/nestoru/gcp-microservices-poc
# https://github.com/nestoru/azure-microservices-poc
# https://github.com/nestoru/aws-microservices-poc
#1. Set in your protected environment specific IaC project the helm vars:
# api-helm/values.yaml
fluentBit:
enabled: true
s3:
accessKey: ***
secretKey: ***
bucket: ***
region: ***
#2. Define a deployment descriptor for a fluent-bit pod to be deployed by helm
# api-helm/templates/fluent-bit-deployment.yaml
{{- if .Values.fluentBit.enabled }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: fluent-bit
labels:
app: fluent-bit
spec:
replicas: 1
selector:
matchLabels:
app: fluent-bit
template:
metadata:
labels:
app: fluent-bit
spec:
serviceAccountName: fluent-bit
containers:
- name: fluent-bit
image: fluent/fluent-bit:latest
volumeMounts:
- name: fluent-bit-config
mountPath: /fluent-bit/etc/
- name: varlog
mountPath: /var/log
readOnly: false # Ensure this is false to allow writing
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: fluent-bit-aws-credentials
key: AWS_ACCESS_KEY_ID
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: fluent-bit-aws-credentials
key: AWS_SECRET_ACCESS_KEY
volumes:
- name: fluent-bit-config
configMap:
name: fluent-bit-config
- name: varlog
hostPath:
path: /var/log
type: DirectoryOrCreate # This ensures the directory is created if it doesn't exist
{{- end }}
#3. Define a configmap kubernetes descriptor containing your settings for log collection for helm to deploy
# api-helm/templates/fluent-bit-configmap.yaml
{{- if .Values.fluentBit.enabled }}
apiVersion: v1
kind: ConfigMap
metadata:
name: fluent-bit-config
data:
fluent-bit.conf: |-
[SERVICE]
Flush 5
Log_Level info
Parsers_File parsers.conf
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_Port 2020
[INPUT]
Name tail
Path /var/log/containers/{{ .Values.appServiceName }}-{{ .Values.majorVersion }}-test*.log
Tag {{ .Values.appServiceName }}.{{ .Values.majorVersion }}
Parser docker
DB /var/log/flb_kube.db
Mem_Buf_Limit 5MB
Skip_Long_Lines On
[OUTPUT]
Name s3
Match {{ .Values.appServiceName }}.{{ .Values.majorVersion }}
bucket {{ .Values.fluentBit.s3.bucket }}
region {{ .Values.fluentBit.s3.region }}
s3_key_format /%Y/%m/%d/{{ .Values.appServiceName }}-{{ .Values.majorVersion }}.log
upload_timeout 1m
use_put_object On
storage.total_limit_size 1G
parsers.conf: |-
[PARSER]
Name docker
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
{{- end }}
#4. Define a kubernetes secret descriptor for the S3 bucket to be deployed by helm
# api-helm/templates/fluent-bit-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: fluent-bit-aws-credentials
type: Opaque
stringData: # Note: Using `stringData` for convenience with plain text values
AWS_ACCESS_KEY_ID: {{ .Values.fluentBit.s3.accessKey }}
AWS_SECRET_ACCESS_KEY: {{ .Values.fluentBit.s3.secretKey }}
#5 Define a kubernetes serviceaccount descriptor to be deployed by helm
# api-helm/templates/fluent-bit-service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: fluent-bit
namespace: {{ .Release.Namespace | default "default" }}

Wednesday, March 20, 2024

Print the content of all zipped files with names containing a regex

#!/bin/bash
#
# The below will print the content of all zipped files with names containing the keyword "application"
# ./print_zip_files_content.sh app.jar '.*application.*'
#
if [ "$#" -ne 2 ]; then
echo "Usage: $0 <zip file path> <regex containing filename>"
exit 1
fi
ZIP_FILE="$1"
REGEX="$2"
# Using unzip -l to list files, then grep to filter by regex. awk to print the 4th column to the end.
# This handles spaces in filenames more gracefully.
unzip -l "$ZIP_FILE" | grep -P "$REGEX" | awk '{print $4}' | while read -r file; do
if [[ ! -z "$file" ]]; then
echo "File $file:"
unzip -p "$ZIP_FILE" "$file"
echo -e "\n"
fi
done

Followers