Tuesday, June 25, 2013

Cleaning up artifactory OSS

Artifactory OSS lacks IMO a clean way to aid on cleaning up unneeded artifacts. But there is also an issue with the algorithm to pick when cleaning artifacts. You cannot go just by which artifacts are old as some old artifacts are still needed and in use.

I have concluded that it is better to share with the team a wiki page (it can be also shared on any SCM like SVN, the important thing is that multiple people can edit the file in a collaborative way) composed of current artifacts and leave the team decides which ones should stay. They just need to remove those they want to keep. After they confirm the job is done you can go ahead and delete them. Below is a step by step procedure to get this done:
  1. Running the below command will list all files from all repos in a file called artifactsToDelete. It uses the current date as "to" to make sure all artifacts are included no matter how old they are, then it extracts from the resulting json string (which contains no new lines === unformatted) the artifact urls. It then removes /api/storage from the resulting URL artifact so we can have the real URL that can be used for deletion purposes.
     curl -XGET -u user:password "http://artifactory.sample.com/api/search/creation?to=$[$(date --date "`date`" +%s)*1000]" | sed 's/uri":"\([^"]*\)"/\n\1\n/g' | grep -o '^http.*/' |  sed 's/api.storage.//g' | sort | uniq > artifactsToDelete
    You could add something like the below to remove those artifacts corresponding to a given package/path:
    grep -v "com/sample"
    Or make sure the list includes only those from a certain package/path:
    grep "com/sample"
  2. Offer the list to the team on a wiki page or as SCM resource. The team should delete the entries they are interested on keeping.
  3. Update artifactsToDelete with the filtered content, create a script to delete them and run it:
    $ vi ./deleteArtifacts.sh 
    #!/bin/bash -ex
    for URL in `cat artifactsToDelete`
      curl -XDELETE -u user:password $URL
    $ chmod +x ./deleteArtifacts.sh 
    $ ./deleteArtifacts.sh 
  4. After it runs go to the Maintenance admin page and run the storage garbage collection.

No comments: