Tuesday, July 23, 2013

Agile team? Did you already script your infrastructure?

It's been two days since Ubuntu Forums and Apple Developer Resources websites have been down. I believe that such big down-term is only related to the fact that the infrastructure is not scripted. Am I wrong?

Recipes are the way to go not only for DR situations but for security reasons as you can see.

Furthermore it is thanks to recipes that we can migrate without fear to new packages or whole OS versions.

Finally it is thanks to recipes that documentation and implementation meet together saving not only a lot of time but a lot of human error as well.

Any change affecting OS or services on top of it should be:
  1. scripted
  2. versioned
  3. applied to servers remotely
That is a culture that should exist in the agile team not only for Linux and Unix but for Windows as well. The times where you rely on documented steps and a sysadmin going through them have passed. It is time to script your infrastructure.

For the record, from http://devimages.apple.com/maintenance:
We’ll be back soon. Last Thursday, an intruder attempted to secure personal information of our registered developers from our developer website. Sensitive personal information was encrypted and cannot be accessed, however, we have not been able to rule out the possibility that some developers’ names, mailing addresses, and/or email addresses may have been accessed. In the spirit of transparency, we want to inform you of the issue. We took the site down immediately on Thursday and have been working around the clock since then. In order to prevent a security threat like this from happening again, we’re completely overhauling our developer systems, updating our server software, and rebuilding our entire database. We apologize for the significant inconvenience that our downtime has caused you and we expect to have the developer website up again soon. If your program membership was set to expire during this period, it has been extended and your app will remain on the App Store. If you have any other concerns about your account, please contact us. Thank you for your patience.
From http://ubuntuforums.org/announce.html:
Ubuntu Forums is down for maintenance There has been a security breach on the Ubuntu Forums. The Canonical IS team is working hard as we speak to restore normal operations. This page will be updated with progress reports. What we know Unfortunately the attackers have gotten every user's local username, password, and email address from the Ubuntu Forums database. The passwords are not stored in plain text, they are stored as salted hashes. However, if you were using the same password as your Ubuntu Forums one on another service (such as email), you are strongly encouraged to change the password on the other service ASAP. Ubuntu One, Launchpad and other Ubuntu/Canonical services are NOT affected by the breach. Progress report 2013-07-20 2011UTC: Reports of defacement 2013-07-20 2015UTC: Site taken down, this splash page put in place while investigation continues. 2013-07-21: we believe the root cause of the breach has been identified. We are currently reinstalling the forums software from scratch. No data (posts, private messages etc.) will be lost as part of this process. 2013-07-22: work on reinstalling the forums continues. If you're using Ubuntu and need technical support please see the following page for support: Finding Help. If you're looking for a place to discuss Ubuntu, in the meantime we encourage you to check out these sites: The Ubuntu subreddit The Ubuntu Community on Google+ Ubuntu Discourse