Wednesday, May 25, 2016

From subversion to git with the help of gitlab

Here are the steps I recently followed to migrate a subversion repo into git:

Thursday, May 19, 2016

Filter output of Linux top command

To filter by process name:
top -cb | grep processName
Or if you want to interact with top just start it using 'top -c', then press "o" and use as filter "COMMAND=processName"

Thursday, April 21, 2016

Streaming and saving your presentation video

I need to write these steps so often that I rather have them handy
  1. go to https://www.youtube.com/
  2. Click on menu | mychannel
  3. Click on video Manager
  4. Click on Live Streaming | Events
  5. Click on New Live event
  6. Title: ${Name of presentation} ${part number}
  7. Select Option "Unlisted" if you want it to be available to only certain people
  8. Click on "go live now"
  9. Share screen and start broadcast
  10. When done stop broadcast
  11. From the video manager get the link and share it with the relevant people

Thursday, April 14, 2016

Upgrade Talend sqlite jdbc library

Talend is still ages behind with the outdated sqlite jar file it uses (sqlitejdbc-v056.jar). While we wait for a fix here is a workaround (using version 6.0.0 here, however I am sure you will figure easily what to do for other versions):
  1. Download latest sqlite-jdbc jar from https://github.com/xerial/sqlite-jdbc/releases
  2. Copy the jar (for example sqlite-jdbc-3.8.11.2.jar) as ~/.m2/repository/org/talend/libraries/sqlitejdbc-v056/6.0.0-SNAPSHOT/sqlitejdbc-v056.jar. Note that you must keep the same old name
  3. Copy the jar as /opt/TOS_DI-20150702_1326-V6.0.0/plugins/org.talend.libraries.jdbc.sqlite3_6.0.0.20150702_1326/lib/sqlitejdbc-v056.jar. Note again that you must keep the same old name. I assume your installation is done in /opt, if not follow the path convention and I am sure you will figure it out.

Sunday, April 10, 2016

Apache proxy to tomcat - Error during SSL Handshake with remote server (AH00898), pass request body failed (AH01097)

An error like the below means that certificates in the proxy server and the target server are not the same or are expired:
[Sun Apr 10 08:13:51.513836 2016] [proxy:error] [pid 32426:tid 140087715120896] [client 192.168.0.5:34425] AH00898: Error during SSL Handshake with remote server returned by /some/path
[Sun Apr 10 08:13:51.513848 2016] [proxy_http:error] [pid 32426:tid 140087715120896] [client 192.168.0.5:34425] AH01097: pass request body failed to 192.168.0.5:8443 (sample.com) from 192.168.0.5 ()
To understand exactly what is going on increase log level temporarily:
LogLevel info proxy:trace5
This will explain what is going on, for example:
[Sun Apr 10 11:45:30.708783 2016] [ssl:info] [pid 26391:tid 140560622925568] [remote 192.168.0.5:8443] AH02004: SSL Proxy: Peer certificate is expired
A one-liner will reveal why. The cert below just expired:
$ echo | openssl s_client -connect 192.168.0.5:8443 2>/dev/null | openssl x509 -noout -dates | grep 'notAfter=.*GMT'
notAfter=Apr 10 12:13:04 2016 GMT
To avoid cluster node certificates expiring at a different time that those from the proxy server use the same for all! To confirm they are the same use a hash to compare they are indeed the same:
$ md5sum /opt/tomcat/certs/my.crt
$ md5sum /opt/tomcat/certs/my.crt
$ md5sum /etc/apache2/certs/my.crt 
$ md5sum /etc/apache2/certs/my.key

Saturday, April 02, 2016

What are the most important Key Performance Indicators (KPI) to be measured?

One of the most commonly asked questions about Project Management is "what are the most important Key Performance Indicators (KPI) I should choose?" The answer as usual is: it depends.
However, assuming that you are building an identified minimal viable product (MVP), for which you will be continuously delivering minimal marketable features (MMF) to create or maintain a minimal marketable product (MMP) or a minimal marketable service (MMS), here is what I think the starting point should be:
  1. Defect Ratio: Because without quality there is not sustainable productivity. To measure it, calculate what percentage of the total MMF are defects both for total open and closed per period MMF to understand the quality of your service or product.
  2. Work In Progress (WIP): Because context switching works against human productivity. To measure it, inspect the cumulative flow diagram (CFD) for increased MMF WIP slope in relationship to closed MMF slope. To be precise, divide the gradients for a ratio. Both slopes should be the same to ensure optimal WIP limits are in place. That means the ratio should be 1. Forcing the Personal WIP limit to 1 is ideal, beyond 3 is chaotic.
  3. Delivery Frequency: Because to avoid entropy you need to deliver as soon as it is done without piling up. To measure it, inspect the CFD for changes in closed-issues slope. The slope should be either constant or increasing, never decreasing. Deployments per period could give you a good measure of how often the company delivers.
  4. Demand versus throughput balance: Because delivery is done through a funnel where the mouth and stem sizes signal the need for adjustment of one of the three iron triangle measures: resources, scope and schedule. To measure it, calculate the open issues divided by target issues. This ratio should be maintained or decreasing, never increasing.
  5. Prioritization time: Because finding out what the customer wants is crucial but time consuming, any time over-spent on this is a nonsense waste. Prioritize based on available slots from the already prioritized backlog queue to keep a small weekly scope-framed value-cost-of-delay-risk multi-var-dimensional analysis. Measure it assigning to each prioritized MMF the time it took in arriving to its choice. Make the prioritization choice based on the results from a radar chart using relevant variables as axis and deciding what is most important based on resulting taxonomies.
  6. Variability: Because predictability is the ultimate goal for continuous delivery of MMF (without continuous delivery the cost of delay and risk goes up). Measure it looking at the control chart for cycle and lead time. The objective is to narrow the standard deviation.
  7. Throughput: Because it is what the business unit can offer in terms of resources and capabilities. Calculate it using the number of MMF completed. Its result is a quantitative measurement of the output.
  8. Productivity: Because the objective is to deliver products or services that are effective (needed) and efficient (optimal use of resources). Calculate effectiveness as the ratio between current hours spent in the value stream and total hours paid. Calculate efficiency as the ratio between expected hours per MMF and your current hours per MMF. The team is more productive when both effectiveness and efficiency are high.
I can't see how a modern organization can achieve sustainable growth and a high capability maturity without using these essential metrics. To me, they are a must to reach operational effectiveness. What is left is Industry comparison. If you have numbers for the below, feel free to share them confidentially with me (nestor.urquiza at gmail). I would like to build Industry standards out of this survey:
Main Industry: ?
Defect Ratio: ?
Personal WIP: ?
Deployments per month: ?
Total MMF open at the end of the month / Total MMF closed per month: ?
Average prioritization time per MMF: ?
MMF cycle time standard deviation: ?
MMF lead time standard deviation: ?
MMF closed per month: ?
Hours spent in value stream / Hours paid: ?
Expected hours per MMF: ?
Current average hours per MMF: ?


Acknowledgement: I wouldn't have dared to think profoundly about metrics without entering the world of lean thinking. I entered this world after reading some W. Edwards Deming literature, getting introduced to TPS, Toyota Kanban and finally studying the blue book from David J. Anderson (a real revelation and source of constant reference for me).

Wednesday, March 30, 2016

AngularJS EvalError: Refused to evaluate a string as JavaScript because 'unsafe-eval'

Following angular documentation for ngCsp looks like just by using the directive is not enough to avoid errors if you use inline scripts. Even after following the guidelines we were still getting randomly the below errors causing angular to stop working and consequently blank content. The reason why it was random was that we had inline javascript coming up randomly in the page:
angular.js:13424 EvalError: Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "default-src 'self' 'unsafe-inline'". at Function.jQuery.extend.globalEval (jquery.js:343) at domManip (jquery.js:5290) at jQuery.fn.extend.after (jquery.js:5456) at domInsert (angular.js:5189) at Object.$provide.$get.$$animateQueue.enter (angular.js:5352) at angular.js:25332 at $$sanitizeUriProvider.$get.node (angular.js:8212) at angular.js:8551 at boundTranscludeFn (angular.js:8350) at controllersBoundTransclude (angular.js:9072)
Inspecting the jquery code it is clear that eval is used:
globalEval: function( code ) {
  var script,
   indirect = eval;

  code = jQuery.trim( code );

  if ( code ) {

   // If the code includes a valid, prologue position
   // strict mode pragma, execute code by injecting a
   // script tag into the document.
   if ( code.indexOf( "use strict" ) === 1 ) {
    script = document.createElement( "script" );
    script.text = code;
    document.head.appendChild( script ).parentNode.removeChild( script );
   } else {

    // Otherwise, avoid the DOM node creation, insertion
    // and removal by using an indirect global eval

    indirect( code );
   }
  }
 }
When angular finds inline scripts it calls the jQuery after() which triggers the globalEval()
      afterElement ? afterElement.after(element) : parentElement.prepend(element);
Clearly a quick way to identify the culprit is just looking into the html content for embedded scripts. In our case Google mod-pagespeed for apache inserted an embedded script tag as shown below:

If the scripts are below an angular directive as in the below example then you will get the error:
<!DOCTYPE html>
<html lang="en" ng-app  ng-csp>
  <head>
    <script type="text/javascript" charset="utf-8" src="jquery.js"></script>
    <script type="text/javascript" charset="utf-8" src="angular.js"></script>
  </head>
  <body>
    <div ng-if="true">
      <script>console.log('foo');</script>
    </div>
  </body>
</html>
If you can live without it then you can remove the script like we did with mod-pagespeed. If not, you will need to include your javascript code from an external source file or avoid loading jquery before angular which will limit what you can do with for example angular.element.

I think Angular should probably just detect if ng-csp is being used to avoid executing inline scripts in such case. It could also come up with a non eval way if there is still a need to run inline scripts just as it does with the "Angular's built-in subset of jQuery, called 'jQuery lite' or jqLite". At a minimum the documentation should state that currently the ng-csp directive will not work if at least if there is inline javascript inside the ng-if directive. This is a problem even if the server sends unsafe-inline like in:
 'Content-Security-Policy:default-src 'self' 'unsafe-inline'"
I submitted this issue as a potential bug for consideration.

Disabling impossible. Removing/Uninstalling mod-pagespeed

The below alone will disable Google mod-pagespeed apache module only temporarily:
sudo a2dismod pagespeed
sudo apachectl configtest
sudo apachectl graceful
For some reason we have seen it enabled back sometime after we run the above. You will need to actually uninstall it. Here is how if you installed it from the debian package in Ubuntu:
sudo sh -c \
  'dpkg -r mod-pagespeed-stable \
  && rm -f /etc/apache2/mods-available/pagespeed* \
  && service apache2 restart'

Followers