fd -t f -0 | xargs -0 -I {} sh -c 'echo "File: {}"; cat {}'
Friday, October 18, 2024
Find content of all relevant files in a project respecting .gitignore
Tuesday, August 06, 2024
Remove server header in IIS to stop disclosing running version(s)
Set attribute removeServerHeader="true" in web.config node configuration."system.webServer".security.requestFiltering.
Thursday, July 18, 2024
When the clock on MAC OSX goes off
To sync your macOS clock and prevent it from drifting after sleep cycles, follow these steps:
- Run this command to sync time:
- Remove the current time settings:
- After reboot, turn on "Set time and date automatically" in System Settings > Date & Time. This should ensure your clock remains accurate.
- Run this command to sync time:
sudo sntp -sS time.apple.com- Go to System Settings > Date & Time and turn off "Set time and date automatically."
- Remove the current time settings:
sudo rm /var/db/timed/com.apple.timed.plist- Reboot the Mac.
- After reboot, turn on "Set time and date automatically" in System Settings > Date & Time. This should ensure your clock remains accurate.
Monday, May 27, 2024
Sunday, April 14, 2024
Minimal kubernetes pods log shipping with fluent-bit
I just released how to ship kubernetes cluster pod logs to Amazon Simple Storage Service (AWS S3) using fluent-bit. Platform engineers should do their best to keep cloud independence. This setup will work for Azure AKS, Google GKE and Amazon EKS. The choice for S3 is just a pick and not a preference. Google Cloud Storage (GCS) or Azure Blob Storage (ABS) will do the job as well (just check fluent-bit docs and adapt the code below for your choice).
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# This setup works seemlessly if you deploy microservices in any of the big 3 cloud providers using terraform and helm as I described in the following github projects: | |
# https://github.com/nestoru/gcp-microservices-poc | |
# https://github.com/nestoru/azure-microservices-poc | |
# https://github.com/nestoru/aws-microservices-poc | |
#1. Set in your protected environment specific IaC project the helm vars: | |
# api-helm/values.yaml | |
fluentBit: | |
enabled: true | |
s3: | |
accessKey: *** | |
secretKey: *** | |
bucket: *** | |
region: *** | |
#2. Define a deployment descriptor for a fluent-bit pod to be deployed by helm | |
# api-helm/templates/fluent-bit-deployment.yaml | |
{{- if .Values.fluentBit.enabled }} | |
apiVersion: apps/v1 | |
kind: Deployment | |
metadata: | |
name: fluent-bit | |
labels: | |
app: fluent-bit | |
spec: | |
replicas: 1 | |
selector: | |
matchLabels: | |
app: fluent-bit | |
template: | |
metadata: | |
labels: | |
app: fluent-bit | |
spec: | |
serviceAccountName: fluent-bit | |
containers: | |
- name: fluent-bit | |
image: fluent/fluent-bit:latest | |
volumeMounts: | |
- name: fluent-bit-config | |
mountPath: /fluent-bit/etc/ | |
- name: varlog | |
mountPath: /var/log | |
readOnly: false # Ensure this is false to allow writing | |
env: | |
- name: AWS_ACCESS_KEY_ID | |
valueFrom: | |
secretKeyRef: | |
name: fluent-bit-aws-credentials | |
key: AWS_ACCESS_KEY_ID | |
- name: AWS_SECRET_ACCESS_KEY | |
valueFrom: | |
secretKeyRef: | |
name: fluent-bit-aws-credentials | |
key: AWS_SECRET_ACCESS_KEY | |
volumes: | |
- name: fluent-bit-config | |
configMap: | |
name: fluent-bit-config | |
- name: varlog | |
hostPath: | |
path: /var/log | |
type: DirectoryOrCreate # This ensures the directory is created if it doesn't exist | |
{{- end }} | |
#3. Define a configmap kubernetes descriptor containing your settings for log collection for helm to deploy | |
# api-helm/templates/fluent-bit-configmap.yaml | |
{{- if .Values.fluentBit.enabled }} | |
apiVersion: v1 | |
kind: ConfigMap | |
metadata: | |
name: fluent-bit-config | |
data: | |
fluent-bit.conf: |- | |
[SERVICE] | |
Flush 5 | |
Log_Level info | |
Parsers_File parsers.conf | |
HTTP_Server On | |
HTTP_Listen 0.0.0.0 | |
HTTP_Port 2020 | |
[INPUT] | |
Name tail | |
Path /var/log/containers/{{ .Values.appServiceName }}-{{ .Values.majorVersion }}-test*.log | |
Tag {{ .Values.appServiceName }}.{{ .Values.majorVersion }} | |
Parser docker | |
DB /var/log/flb_kube.db | |
Mem_Buf_Limit 5MB | |
Skip_Long_Lines On | |
[OUTPUT] | |
Name s3 | |
Match {{ .Values.appServiceName }}.{{ .Values.majorVersion }} | |
bucket {{ .Values.fluentBit.s3.bucket }} | |
region {{ .Values.fluentBit.s3.region }} | |
s3_key_format /%Y/%m/%d/{{ .Values.appServiceName }}-{{ .Values.majorVersion }}.log | |
upload_timeout 1m | |
use_put_object On | |
storage.total_limit_size 1G | |
parsers.conf: |- | |
[PARSER] | |
Name docker | |
Format json | |
Time_Key time | |
Time_Format %Y-%m-%dT%H:%M:%S.%L | |
Time_Keep On | |
{{- end }} | |
#4. Define a kubernetes secret descriptor for the S3 bucket to be deployed by helm | |
# api-helm/templates/fluent-bit-secret.yaml | |
apiVersion: v1 | |
kind: Secret | |
metadata: | |
name: fluent-bit-aws-credentials | |
type: Opaque | |
stringData: # Note: Using `stringData` for convenience with plain text values | |
AWS_ACCESS_KEY_ID: {{ .Values.fluentBit.s3.accessKey }} | |
AWS_SECRET_ACCESS_KEY: {{ .Values.fluentBit.s3.secretKey }} | |
#5 Define a kubernetes serviceaccount descriptor to be deployed by helm | |
# api-helm/templates/fluent-bit-service-account.yaml | |
apiVersion: v1 | |
kind: ServiceAccount | |
metadata: | |
name: fluent-bit | |
namespace: {{ .Release.Namespace | default "default" }} |
Wednesday, March 20, 2024
Print the content of all zipped files with names containing a regex
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# | |
# The below will print the content of all zipped files with names containing the keyword "application" | |
# ./print_zip_files_content.sh app.jar '.*application.*' | |
# | |
if [ "$#" -ne 2 ]; then | |
echo "Usage: $0 <zip file path> <regex containing filename>" | |
exit 1 | |
fi | |
ZIP_FILE="$1" | |
REGEX="$2" | |
# Using unzip -l to list files, then grep to filter by regex. awk to print the 4th column to the end. | |
# This handles spaces in filenames more gracefully. | |
unzip -l "$ZIP_FILE" | grep -P "$REGEX" | awk '{print $4}' | while read -r file; do | |
if [[ ! -z "$file" ]]; then | |
echo "File $file:" | |
unzip -p "$ZIP_FILE" "$file" | |
echo -e "\n" | |
fi | |
done |
Sunday, March 10, 2024
Securely sharing the content of your MAC OSX ~/.zshrc and other profile configuration files
cat ~/.zshrc | sed -E "s/(export[^=]*=).*/\1***/"
Subscribe to:
Posts (Atom)