Principles of Software Flow


Creating a Drools Playground with Fitnesse

Playground at Fuji-Hakone-Izu National Park

/Playground at Fuji-Hakone-Izu National Park

Who sucks all the joy out of Drools?

There it is, this exciting new technology that I’m keep to play with.  I’m eager to play with it but the documentation makes me wade through countless pages about  building, deploying, utilizing and running.  Yawn, that’s no fun.

When it comes to learning I’m with Forth creator’s Charles Moore “Fun Down” approach:

I’ll do the thing that’s most fun in order to get into the problem. If I have to clean up all those details later, that’s the price I pay.

Ian Brodies’ “Thinking Forth” page 91 

I don’t want to spend ages setting the thing up.  I just want to try stuff.  See what works, what doesn’t.  I want to suck it and see.  All this seriousness just sucks.

I’m not going to be deterred by it.  As a general rule anything that tries to be fun is dull and anything that tries this hard to be serious must be great fun.  The name, Drools, shows the playfulness of the creators.  The communities rejection of the JBoss rules brand shows that the playfulness lives on.  The frowns and tuts tuts of some cannot discourage it.

Consider, for example, Marc Proctor’s Pong Game in 13 minutes on You Tube: https://www.youtube.com/watch?v=Omj4PR3v-nI

Build_Pong_in_13_minutes_using_JBoss_Drools_-_YouTube

Here we see the playfulness.  It was an opportunity to show just how disruptive a technology Drools can be, breaking all the rules in order to achieve something unexpected.  As was explained further in the comments:

Mark Proctor n reply to Rick Kilcoyne:

I hope these videos show that rule engines are more than just “if” statements, and the declarative programming model is quite powerul and more general puprose than people probably realise.

Frowns and tut tuts?  Of course.  Consider the following comments from You Tube:

detour98:
Mark, what you’ve just showed is one of the most insanely idiotic things I have ever seen. At no point in your copy pasting, alt tabbing, incoherent code were you even close to anything that could be considered a 13 minute build of anything. Everyone in this video comments is now dumber for having spent 14:39min of their lives watching it. I award you no points, and may God have mercy on your soul.

lukeyd13:

This shit doesnt compile at all?!!!

Does any of that distract from how fascinating the example is?  Not at all.  It is a simple example that provides a gilmpse of how things could be done differently.  This isn’t lost on everybody:

Rick Kilcoyne in reply to Mark Proctor:
Thanks! And well done.

Paul St. Amant:
You are awesome. I thought I was watching an episode of Lost. And I loved Lost.

MeanMachine11111:
super amazing

While at work I will use Drools in an appropriately serious manner for proven use cases.  At home I’m going to play with Drools and have fun breaking some rules.  I’m going to follow the example set by Mr Proctor and his 70s video game.

In order to enjoy Drools I’m going to have to create a playground for myself.  Somewhere I can safely mess about with the core engine without having my flow disrupted by the having to wait for builds and deployments.

For building a playground my tool of choice is Fitnesse.

Getting Playful with Fitnesse

In contrast to Drools chapters here is what you need to do to get up and running with Fitnesse:

It really takes very little time and effort to get FitNesse running on your machine.

Click on the most recent fitnesse-standalone.jar file.

  • If this is a brand new installation, just put it in some convenient empty directory.
  • If this is an update, then copy it on top of the old fitnesse(-standalone).jar in your current installation.

Type java -jar fitnesse-standalone.jar

  • It will ask you to be patient as it installs or updates.
  • Your shell should respond with something like this:
        FitNesse (v20130228) Started...
         	port:              8001
         	root page:         fitnesse.wiki.FileSystemPage at ./FitNesseRoot
         	logger:            none
         	authenticator:     fitnesse.authentication.PromiscuousAuthenticator
         	page factory:      fitnesse.responders.PageFactory
         	page theme:        bootstrap
         	page version expiration set to 14 days.

Start up a browser and go to http://localhost

The main FitNesse screen should come up. That’s it. You’re ready to start using FitNesse.

The Download and Install Instructions for Fitnesse

That’s the way to do it!

Fitnesse is like Chess.  It takes no time at all to get started but a whole lifetime to master.

I can understand why Drools has such powerful features for developing and deploying with distributed teams and complex production architecture.  It’s the secret sauce being used on mission critical production deployments.  There are teams supporting production who are grateful for this sophistication.

But it comes at a cost.  To realise the true benefits of this new programming paradigm we have to change the way we think.  We cannot do that when we are struggling with the complexity.  To remodel our minds we must be able to play with something fun and engaging.

How can Fitnesse help me to achieve that?

Let me show you.

First we will get Fitnesse and Eclipse working together to provide us with a fast and simple workflow.

I will take you through these steps carefully, so you can get Fitnesse up and running on your machine.

Then we will create the User Guide’s sprinkler example in Eclipse.

Finally I’ll refactor these examples into Fitnesse Fixtures.

I’ll move quickly through these steps to give you an idea of what I’m doing.  If you’re new to Fitnesse don’t expect to follow along.  The refactorings are based on a knowledge of some important Fitnesse concepts that I will explain in later posts.

At the end I’ll share the full implementation so that you should be able to get it up and running.  This will lay the foundation for what is to follow – a deep dive into the Drools core.

This set up isn’t intended for production.  It’s for learning about Drools through play.  It can be useful during the early stages of development so that the team can get into the problem without being distracted.  It is especially useful when you have developers who need to change their way of thinking because it removes the familiar Java IDE.

Getting Fitnesse and Eclipse to Play Nicely Together

Fitnesse goes first

Follow the simple instructions above to get Fitnesse up and running.  If you get an error because the port is already in use then use the -P option to specify another one.  See the cheat sheet for details of the command line options.

When you browse to localhost you’ll see the front page.

Fitnesse Start Page

Fitnesse is a typical Wiki, so the first thing you want to do is create a page to work from.

Just hit Edit and insert this line at the top of the page:

[[Drools Playground][DroolsPlayground.StartHere]]

This will give you a link with question mark.  Click on the question mark to create the new page.

Fitnesse Start Page

You can worry about what you want to write here later.   For now just keep the default text:

!contents -R2 -g -p -f -h

If you want to know what all those options are doing take a look at the User Guide.

Save the page and it will be empty because there are not contents yet.  Lets do something about that!

Hover over the Add button and select the Test Page option.  A new test page will be created.

Fitnesse Start Page

Call the page “HelloWorld”.  It has to be a wiki word with at least two capital letters.  Then copy in the following content.

!|Import|
|uk.co.flowcoding.playground.drools|
|Hello World|
|toString?|
|Hello|
|World|

Hit save and you’ll be returned to the StartHere page showing our new HelloWorld in the table of contents.

Fitnesse Start Page

Follow the HelloWorld link to see the page we created.

Fitnesse Start Page

If you compare the wiki markup with the formatted page you shouldn’t have much difficulty seeing how the two tables are defined.  The question is what are those tables for?  What do they do?  Allow me to explain.

The first table is a package import.

Fitnesse Start Page

It imports Java packages so they can be used in other tables.  These other tables are called fixtures.  Read the full details in the Fitnesse User Guide.

The second table is a fixture.

Fitnesse Start Page

It’s name is mapped to a Java class in the imported package.  The column heading maps to a method.  The question mark tells us it is a method.  Without the question mark it would map to a field.

Hello World will be a Row Fixture.  The User Guide explains:

RowFixture

RowFixture tests dynamic lists of objects. It will compare the expected list (FitNesse table) with the actual result (from fixture code) and report any additional or missing items.

Table Format

The first row of the table is the fixture class name. The second row describes the structure of objects in the list (the properties or methods that you want to verify). All rows after that describe expected objects in the array.

!|RowFixtureTest|
|name|post code|
|John Smith|SW4 66Z|
|Michael Jordan|NE1 8AT|

Hit the test button and see what happens.

Fitnesse Start Page

If fails!

At the top of the page we get a report similar to JUnit informing us that we have 1 exception.

The exception is “Could not find fixture: HelloWorld”.  Notice that the space has been taken out of the table name to turn it into a Java Class name.  Fitnesse has searched it’s classpath for the HelloWorld class and failed with an exception.

To fix this we just need to add our HelloWorld class to the Fitnesse classpath.

Before we can do that we will have to write the HelloWorld class.  For that we will need Eclipse.

Now it’s Eclipse’s turn

I’m assuming you already know Eclipse so I’ll move fast here.

Fitnesse Start Page

  1. Fire up Eclipse and create a new Java Project called Drools Playground.
  2. Add the fitnesse-standalone jar from your Fitnesse installation project to the folder.
  3. Create a package called uk.co.flowcoding.playground.drools.
  4. Create the following class in that package:
    package uk.co.flowcoding.playground.drools
    import fit.RowFixture
    public class HelloWorld extends RowFixture {
        @Override
        public Class getTargetClass() {
            return String.class
        }
        @Override
        public Object[] query() throws Exception {
            return new String[] {"Hello", "World"}
        }
    }
    

This is a RowFixture.  The Fitnesse User Guide explains:

Fixture class

The fixture class should extend fit.RowFixture and override the following two methods:

getTargetClass — returns the Type or Class object representing the type of objects contained in the array.

query — returns the actual array of objects to be verified.

Now all we need to do is to tell Fitnesse where this code is.

Now Fitnesse and Eclipse Together

We need to add the binary class files to the Fitnesse classpath.

Take a look at the project properties to find out where that is..

First look at the Resource dialogue to see where the project folder is:

Fitnesse Resource

To find out where the binary class files are being built look at the Source tab of the Java Build Path dialogue.

Eclipse Source Properties

On my machine the files can be found in “/home/ged/workspace/DroolsPlayground/bin”.

Fitnesse Start Page

So I set the classpath in Fitnesse to “/home/ged/workspace/DroolsPlayground/bin”.

To do this just edit the Start Here page and add the following to the to bottom of the page:

!path /home/ged/workspace/DroolsPlayground/bin

Now every page below StartHere will share this class path.

Returning to the HelloWorld test page and hitting Test I get to see the TDDers favourite colour: green.

flow

The Software Flows

You can see how the Row Fixture works.  In the HelloWorld class you can see a string array with two items:

    @Override
    public Object[] query() throws Exception {
        return new String[] {"Hello", "World";};
    }

In the HelloWorld table we see the same two values.

!|Import|
|uk.co.flowcoding.playground.drools|
|Hello World|
|toString?|
|Hello|
|World|

Since they both match, everything turns green. Now I alt-tab to Eclipse and change the array:

        return new String[] {"Hello", "Big"; "Wide", "World"};

Then I alt-tab to Fitnesse and hit Test again and the green bar has gone. It’s red, with a nice helpful navigator to help me find my failures. flow

Hello and World still match, but now we also have two surplus values: Wide and Big. So I hit edit, add the two values, and test again. Yeah, green bar is back! flow You can see that Row Fixture doesn’t care about the order. If order was important I would have used an Array Fixture. No plugins to install, no need to mess with Maven. This a nice simple integration based on basic fundamentals.  Fitnesse and Eclipse share a folder and it just works.  I am no longer a slave to my tools – If anything every goes wrong I have a chance of figuring out why and fixing it.  In the process I can only learn more.

Elcipse-Fitnesse-Workflow

If you’re new to Fitnesse I hope this gives you a taste for what it can do. It provides just a few simple features but once you master them there seems no limit to what you can achieve with them.  Uncle Bob shows by example the power of clean and simple can do.

I believe that I will find something just a simple beating at the heart of Drools Expert.

Drools Joins In

with Drools I’m going to introduce the sprinkler example from chapter 6 of the Drools 6.1 documentation.

Just in Eclipse to Begin With

Before tackling the example I follow the User Guide to set up the example.  The following describes the process of trial and error I followed rather than providing a neat set of instructions:

  • I create 4 classes: Room, Sprinkler, Fire and Alarm.
  • I use the Eclipse ‘Encapsulate Field’ re-factoring to create the getters and setters.
  • I create a new folder in the project root called “resources”
  • I create a file in that folder called SprinklerRules.drl.
  • I copy and paste all the example rules from Chapter 6 of the 6.1.0.Final User Guide.
  • At the top of the file I add a package import to match the java classes.
  • I create a new folder under resources called META-INF.
  • I create a file in that folder called kmodule.xml
  • I copy and paste the minimal kModule configuration from the User Guide.
  • In the same package as the 4 classes I create a RunRules class with a public static void main method.
  • I copy the demonstration code from the UserGuide into the main method.
  • Eclipse tells me that this code expects my classes to have constructors, so I add them.

This now gives me the following source code:

In src/…

  • RunRules.java
    • package droolsdocs.chapter6.statefulsession.sprinklers;
      
      import java.util.HashMap;
      import java.util.Map;
      
      import org.kie.api.KieServices;
      import org.kie.api.runtime.KieContainer;
      import org.kie.api.runtime.KieSession;
      import org.kie.api.runtime.rule.FactHandle;
      
      public class RunRules {
      
       public static void main ( String[] args ) {
        //Get a KieSession
        KieServices kieServices = KieServices.Factory.get();
        KieContainer kContainer = kieServices.getKieClasspathContainer();
        KieSession ksession = kContainer.newKieSession();
       
        //Set Up The Rooms and Store them in a Hashmap
        String[] names = new String[]{"kitchen", "bedroom", "office", "livingroom"};
      
        Map<String,Room> name2room = new HashMap<String,Room>();
      
        for( String name: names ){
         Room room = new Room( name );
         name2room.put( name, room );
         ksession.insert( room );
         Sprinkler sprinkler = new Sprinkler( room );
         ksession.insert( sprinkler );
        }
      
        //Fire the Rules
        ksession.fireAllRules();
       
        //Start Some Fires
        Fire kitchenFire = new Fire( name2room.get( "kitchen" ) );
        Fire officeFire = new Fire( name2room.get( "office" ) );
      
        FactHandle kitchenFireHandle = ksession.insert( kitchenFire );
        FactHandle officeFireHandle = ksession.insert( officeFire );
      
        //Fire the Rules 
        ksession.fireAllRules();
      
        ksession.delete( kitchenFireHandle );
        ksession.delete( officeFireHandle );
      
        ksession.fireAllRules();
       
       }
      }
      
  • Room.java
    • package droolsdocs.chapter6.statefulsession.sprinklers;
      
      public class Room {
       private String name;
      
       public Room(String name) {
       setName(name);
       }
      
       public String getName() {
       return name;
       }
      
       public void setName(String name) {
       this.name = name;
       }
      }
      
  • Sprinkler.java
    • package droolsdocs.chapter6.statefulsession.sprinklers;
      
      public class Sprinkler {
       private Room room;
       private boolean on;
       
       public Sprinkler(Room room) {
       setRoom(room);
       }
       public Room getRoom() {
       return room;
       }
       public void setRoom(Room room) {
       this.room = room;
       }
       public boolean isOn() {
       return on;
       }
       public void setOn(boolean on) {
       this.on = on;
       }
      }
      
  • File.java
    • package droolsdocs.chapter6.statefulsession.sprinklers;
      
      public class Fire {
       private Room room;
      
       public Fire(Room room) {
       setRoom(room);
       }
      
       public Room getRoom() {
       return room;
       }
      
       public void setRoom(Room room) {
       this.room = room;
       }
      }
      
  • Alarm.java
    • package droolsdocs.chapter6.statefulsession.sprinklers;
      
      public class Alarm {
       private String name;
      
       public String getName() {
       return name;
       }
      
       public void setName(String name) {
       this.name = name;
       }
      }
      

in resources/

  • SprinklerRules.drl
    • package droolsdocs.chapter6.statefulsession.sprinklers;
      
      rule "When there is a fire turn on the sprinkler"
      when
       Fire($room : room)
       $sprinkler : Sprinkler( room == $room, on == false )
      then
       modify( $sprinkler ) { setOn( true ) };
       System.out.println( "Turn on the sprinkler for room " + $room.getName() );
      end
      
      rule "When the fire is gone turn off the sprinkler"
      when
       $room : Room( )
       $sprinkler : Sprinkler( room == $room, on == true )
       not Fire( room == $room )
      then
       modify( $sprinkler ) { setOn( false ) };
       System.out.println( "Turn off the sprinkler for room " + $room.getName() );
      end
      
      rule "Raise the alarm when we have one or more fires"
      when
       exists Fire()
      then
       insert( new Alarm() );
       System.out.println( "Raise the alarm" );
      end
      
      rule "Cancel the alarm when all the fires have gone"
      when
       not Fire()
       $alarm : Alarm()
      then
       delete( $alarm );
       System.out.println( "Cancel the alarm" );
      end
      
      rule "Status output when things are ok"
      when
       not Alarm()
       not Sprinkler( on == true ) 
      then
       System.out.println( "Everything is ok" );
      end
      
  • META-INF/kmodule.xml
    • <?xml version="1.0" encoding="UTF-8"?>
      <kmodule xmlns="http://jboss.org/kie/6.0.0/kmodule"/>
      

Before the code will run I need to provide the JARs that the code depends upon.  I discover these trough trial and error.

  • I create a lib folder alongside src and resources.
  • I download Drools 6.1 distribution.
  • I copy folders from the distribution bin into my project’s lib folder and add them to the Eclipse build path.

Within the Drools 6.1 distribution there is a bin folder full of Jars.  I use FindJars to find out what Jars I need.  First I add what is needed to import KieServices, KieContainer and KieSession.  Then I run the code and find the jars needed to resolve the class not found exceptions.

Here is my final list of Jars:

Still more than I would like.  I shall come back to these in a later post.  For now, however, I press on.

With code and Jars in place I can run the code and I’m rewarded with the expected result:

  • SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
    SLF4J: Defaulting to no-operation (NOP) logger implementation
    SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
    Everything is ok
    Turn on the sprinkler for room kitchen
    Turn on the sprinkler for room office
    Raise the alarm
    Turn off the sprinkler for room kitchen
    Turn off the sprinkler for room office
    Cancel the alarm
    Everything is ok
    

Bringing in Fitnesse

Now to take this code an translate it into a set of Fitnesse Fixtures.

Creating Fitnesse Fixtures should be treated as the Creation of a Domain Specific Language.  We want to break our code down into the high level tasks being completed, and give each of those a Fixture.   Then we can mix and match those fixtures to create new scenarios without having to write code.

My goal is to make the Fitnesse DSL as expressive as possible, allowing me to implement all the scenarios I’m interested in without having to write any scenario specific code.  To reach that goal I need to take one step at a time, so I start with Run Rules.  I wan’t to eliminate that method all together, since it is full of scenario specific code.

So let’s break the RunRules class up using the basic Extract Method refactoring.

Doing this gives me the following

package droolsdocs.chapter6.statefulsession.sprinklers;

import java.util.HashMap;
import java.util.Map;

import org.kie.api.KieServices;
import org.kie.api.runtime.KieContainer;
import org.kie.api.runtime.KieSession;
import org.kie.api.runtime.rule.FactHandle;

public class RunRules {

 private static KieSession ksession;
 
 private static Map<String,Room> name2room = new HashMap<String,Room>(); 
 
 private static void setUpKieSession() {
  KieServices kieServices = KieServices.Factory.get();
  KieContainer kContainer = kieServices.getKieClasspathContainer();
  ksession = kContainer.newKieSession();
 }
 
 private static void AddRooms(String... names) {
  for( String name: names ){
   Room room = new Room( name );
   name2room.put( name, room );
   ksession.insert( room );
   Sprinkler sprinkler = new Sprinkler( room );
   ksession.insert( sprinkler );
  }
 }
 
 private static void fireTheRules() {
  ksession.fireAllRules();
 }
 
 private static FactHandle startFireInRoom(String name) {
  Fire fire = new Fire( name2room.get( name ) );
  return ksession.insert( fire );
 }
 
 private static void stopFire(FactHandle fire) {
  ksession.delete(fire);
 }
 
 public static void main ( String[] args ) {
 
 setUpKieSession();
  AddRooms("kitchen", "bedroom", "office", "livingroom");
  fireTheRules();
  FactHandle kitchenFireHandle = startFireInRoom("kitchen");
  FactHandle officeFireHandle = startFireInRoom("office");
  fireTheRules();
  stopFire(kitchenFireHandle);
  stopFire(officeFireHandle);
  fireTheRules();
 }
}

There are a couiple of awkward features I want to smooth out here:

  • The Room2Names map
  • The FactHandles required to stop a fire.

To tackle these I refactor some more.

First of all I get rid of the Rome2Names map by making the FactHandle the only object reference I pass around.  I use the getObject(FactHandle) method to retrieve the Room objects.

Then I replace all the FactHandles with their External forms and use a DisconnectedFactHandle.  Now all I pass around are strings.

Now I am ready to move the code to Fixtures.

Playtime!

I can now share with you the files for the working Drools playground.  Over the next couple of weeks I’ll share my approach with Fitnesse as well as my discovery of the Drools core in a serious of blog posts.  You’ll be glad to hear that those blog posts will be much shorter.

I’m currently setting up the project on GitHub.  I’ll update this post with the details when it’s available.


Playing the Fool With Drools

The Fool is… the childlike ability to tune into the inner workings of the world.


Learning through play

I want to get to know Drools properly.  The core fascinates me but it always seems to be obscured from view.  I want to get past the layers of sophistication and tune into the inner workers of the rules engine.

So I’m going to take a different approach to learning.  I’m going to fool around with Drools and learn through play.

I’m going to perform playful experiments with five key features of Drools that interest me.

The Drools Rule Engine

Over the next few weeks I’m going to post some articles playing with the following features.  I’ll try to take the feature and use them in a way that is different from the usual examples.  I doubt I’ll create anything useful.  I may learn a few useful things.  Only one thing is certain: I’m going to enjoy myself.

1.Declaring Classes

It’s possible to declare new Classes within the DRL file.

The reference guide gives the following example:

declare Person
    name : String
    dateOfBirth : java.util.Date
    address : Address
end

Which is the equivalent to this Java code:

public class Person implements Serializable {

    private String name;

    private java.util.Date dateOfBirth;

    private Address address;

    // empty constructor

    public Person() {...}

    // constructor with all fields 

    public Person( String name, Date dateOfBirth, Address address ) {...}

    // if keys are defined, constructor with keys

    public Person( ...keys... ) {...}

    // getters and setters

    // equals/hashCode

    // toString

 

As you can see, the Drools syntax is far more concise than the full blown Java.  It also saves me having to worry about implementing hashCode and equals.

2. Traits

Traits allow you to implement a form of multiple inheritance.

The Drools examples include a demonstration of traits that extends the Person class.


declare Parent
 @format ( trait )
 children : List
end 

declare Father
end

declare Mother
end

rule xxx when
 $person : Person()
 exists ( Person() from $person.fields[ "children" ] )
then
 don( $person, Parent.class, true );
end 

rule xxx when
 $parent : Parent( gender == "male" )
then
 don( $per, Father.class, true );
end 

rule xxx when
 $parent : Parent( gender == "female" )
then
 don( $per, Mother.class, true );
end 

rule xxx when
 $father : Father()
 Today( this == Sunday )
then
 System.out.println( $father + ": It's your day off" );
end

I’d be interested to see what I could do with that particular feature.  It appears to allow a form of specialisation by constraint.

3. Streams

Fusion introduces support for Streams.  This allows streams of events to be processed, like in this example:

rule "authorize withdraw"
when
    WithdrawRequest( $ai : accountId, $am : amount ) from entry-point "ATM Stream"
    CheckingAccount( accountId == $ai, balance > $am )
then
    // authorize withdraw
end

I wonder if I can use Streams to implement Flow Based Programming using Drools?

4. Queries

Queries seem to be the easiest way to get data out of working memory.  They are clear and easy to read, like this example:

query "people over the age of 30" 
    person : Person( age > 30 )
end

Behind this simplicity lurks something far more complex. Consider this recursive example:

declare Location
    thing : String 
    location : String 
end
 
query isContainedIn( String x, String y ) 
    Location(x, y;)
    or 
    ( Location(z, y;) and ?isContainedIn(x, z;) )

What is even more interesting is the explanation given for the ‘?’ prefix:

The ‘?’ symbol means the query is pull only, once the results are returned you will not receive further results as the underlying data changes.

Does this mean that you can have push queries that do continue to provide results as the underlying data changes?

5. Backward Chaining

What on earth is Backward Chaining?   The documentation explains:

Drools supports unification for derivation queries, in short this means that arguments are optional. It is possible to call queries from Java leaving arguments unspecified using the static field org.drools.core.runtime.rule.Variable.v – note you must use ‘v’ and not an alternative instance of Variable. These are referred to as ‘out’ arguments. Note that the query itself does not declare at compile time whether an argument is in or an out, this can be defined purely at runtime on each use.

No.  I’m none the wiser.

Perhaps the example will makes things clearer:

package org.drools.examples.backwardchaining;

import org.drools.examples.backwardchaining.Location;

dialect "mvel"

query isContainedIn( String x, String y )
 Location( x, y; )
 or
 ( Location( z, y; ) and isContainedIn( x, z; ) )
end

rule "go" salience 10
when
 $s : String( )
then
 System.out.println( $s );
end


rule "go1"
when
 String( this == "go1" )
 isContainedIn("Office", "House"; )
then
 System.out.println( "office is in the house" );
end

rule "go2"
when
 String( this == "go2" )
 isContainedIn("Draw", "House"; )
then
 System.out.println( "Draw in the House" );
end

rule "go3"
when
 String( this == "go3" )
 isContainedIn("Key", "Office"; )
then
 System.out.println( "Key in the Office" );
end

rule "go4"
when
 String( this == "go4" )
 isContainedIn(thing, "Office"; )
then
 System.out.println( "thing " + thing + " is in the Office" );
end

rule "go5"
when
 String( this == "go5" )
 isContainedIn(thing, location; )
then
 System.out.println( "thing " + thing + " is in " + location );
end

I can’t see anything special here. Perhaps there’s something in the java:

package org.drools.examples.backwardchaining;

import org.kie.api.KieServices;
import org.kie.api.runtime.KieContainer;
import org.kie.api.runtime.KieSession;

import java.text.SimpleDateFormat;

public class HouseOfDoomMain {

 public static void main(String[] args) throws Exception {
 KieContainer kc = KieServices.Factory.get().getKieClasspathContainer();
 KieSession ksession = kc.newKieSession( "HouseOfDoomKS");

 ksession.insert( new Location("Office", "House") );
 ksession.insert( new Location("Kitchen", "House") );
 ksession.insert( new Location("Knife", "Kitchen") );
 ksession.insert( new Location("Cheese", "Kitchen") );
 ksession.insert( new Location("Desk", "Office") );
 ksession.insert( new Location("Chair", "Office") );
 ksession.insert( new Location("Computer", "Desk") );
 ksession.insert( new Location("Draw", "Desk") );

 ksession.insert( "go1" );
 ksession.fireAllRules();
 System.out.println("---");

 ksession.insert( "go2" );
 ksession.fireAllRules();
 System.out.println("---");

 ksession.insert( "go3" );
 ksession.fireAllRules();
 System.out.println("---");

 ksession.insert( new Location("Key", "Draw") );
 ksession.fireAllRules();
 System.out.println("---");

 ksession.insert( "go4" );
 ksession.fireAllRules();
 System.out.println("---");

 ksession.insert( "go5" );
 ksession.fireAllRules();
 }
}

Is it just me? What is going on? What is this Backward chaining actually doing?

I’m going to have some fun finding out.


Is The Anti Matter Principle Useful?

“Attend to Folk’s Needs.”

I must say that when I first read Bob Marshall’s Anti Matter Principle I was somewhat underwhelmed.

It was announced with a fanfare of rhetoric. Somewhat mismatched rhetoric, I thought.

I don’t like the name. It points to the underlying rules of the universe, to the science of the fundamental particles from which our world is constructed.

The name promises much. So does the rhetoric:

“The Antimatter Principle – the only principle we need for becoming wildly effective at collaborative knowledge work.”

The Anti Matter Principle

I imagine men in white coats analysing the data from their particle accelerator for decades, distilling the results into a principle for everything.

And then the principle is revealed:

“Attend to folks needs”

ibid

 

And I thought to myself: “What a dull principle.”

The white coats have gone and now the men sit around the campfire strumming guitars, gazing into the flames and singing along.

We need more science!


Continue reading this entry »


Are all Models Wrong?

One of my favourite quotes comes from the accidental statistician George E. P. Box

Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful.

http://en.wikiquote.org/wiki/George_E._P._Box#An_Accidental_Statistician.2C_2010

It was interesting to find a blog article seeking to disprove this assertion.

Here is the crux of his argument:

Suppose Model A states, “X will occur with a probability that is greater than 0 or less than 1.” And let Model B state that “X will occur”, which of course is equivalent to “X will occur with probability 1″ (I’m using “probability 1″ in its plain-English, and not measure-theoretic, sense).

 

Now, if X does not occur, Model B has been proved false. The popular way to say it is that Model B has been falsified. If X does occur, Model B has been proved true. It has been truified, if you like.

 

How about Model A? No matter if X occurs or not, Model A has not been falsified or truified. It is impossible for Model A to be falsified or truified.

http://wmbriggs.com/blog/?p=1906

Now, I am not a statistician.  However, I reject this argument as wrong (although I accept it is probably true.)

The reason why is explained in the comments:

I had the good fortune to study with Dr. Box, and I’m afraid you’ve misconstrued [h]is aphorism. You have somehow managed to conflate “All models are wrong” with “All models are false” and then went on your merry way skewering your strawman.

 

I can assure you from first hand interaction with Dr. Box, that “All models are wrong” means simply, “All models have error”. In the silly example you state, the “unfalsifiable” Model A isn’t really even a model. 

ibid

However, while the argument is wrong, I feel it is still useful.

When Matt Briggs presents Model A

“X will occur with a probability that is greater than 0 or less than 1.” 

He is actually saying “we now that this model is wrong.”

If the probability were 0.5 he would be saying “Half the time this model is wrong, half the time it is right.”

He is still saying that the model is wrong.  He is just quantifying the number of times he expects it be prove wrong.

Which is exactly what George Box was talking about.  

how wrong do they have to be to not be useful.

Is being wrong half the time good enough to be useful?  How about sixty percent?

At this point the precision of the language does not suit the levels of uncertainty we are dealing with when people like myself, non statisticians, are talking.

Instead we are better off using less precise terms such as “unlikely”, “more often than not” or “rarely.”

Michael Jackson captures this distinction in his Problem Frame approach by distinguishing between formal and informal descriptions.

He explained the distinction back in 1998:

In any system of this kind it is important to recognise that the machine is completely formal, while the world is almost invariably mostly informal. The machine has been carefully constructed so that its fundamentally
informal physical nature has been tamed and brought under control.

http://www.jacksonworkbench.co.uk/stevefergspages/papers/jackson–a_discipline_of_description.pdf

In software the machines we build are always models and nothing more.  All models are wrong.  Software will always have bugs.

Why does any of this matter?

Because lately it has become fashionable to reject formal disciplines and practices that have proven useful simply because they can be proven to be wrong.  The response is to be become completely informal.

A good example of this is the NoEstimates movement.

Instead of depending on an accurate estimate for predictability we can take away the unknowns of cost and delivery date by making them… well, known. 

http://neilkillick.com/2013/01/31/noestimates-part-1-doing-scrum-without-estimates/

I’m not criticising Neil Killick here.  He makes a good argument.  The problem I have is with the people who have tried to use his “no estimates” aphorism as a principle or even, sometimes, as fact.

Are estimates always wrong?  Yes.

Are estimates sometimes useful?  Yes.

The problem is that we need to know just how wrong our estimate are if we are to know when they are useful.


What Does It Mean to Move On From Agile?

What do I mean when I say I’m moving on from Agile?

Am I abandoning the values and principles?

Of course  not, because these were my values and principle prior to the Agile Manifesto.

This way of thinking was around for decades before the term “Agile” was coined.  If you don’t believe me then read Brodie’s Thinking Forth (1984).

These principles were not a revelation received at Snowbird.  It was the common core values that all of the participants shared.

The Agile Manifesto was written in February of 2001, at a summit of seventeen independent-minded practitioners of several programming methodologies. The participants didn’t agree about much, but they found consensus around four main values.

http://www.agilealliance.org/the-alliance/the-agile-manifesto/

Let’s just put this conference into perspective.  17 developers got together, argued a lot and then finally came up with a core set of values.

I know that 17 developers actually managing to agree on something is a rare and precious thing, but please have a sense of perspective.  Please stop quoting the manifesto like it’s scripture.

Does it mean that I’ll stop using incremental development?

Of course not, because I was already using incremental development prior to the Agile Manifesto.  First I did RAD, then DSDM.  Since Arie van Bennekum happened to be there DSDM gets call itself Agile.

Is Glib’s EVO Project Management method Agile?  No, because he wasn’t there.

What if Ari van Bennekum had missed his flight?  Would DSDM still be considered Agile?  Is Agile just an accident of scheduling?  Yet this label, Agile, is used as if it has some deep, significant meaning.

For many years Agile served as a useful signal.  When I started on a new team I could be confident that the other members talking about Agile would share my values and perspective.  The term “Agile” served as a useful placeholder for those values and perspectives.  It was a perspective that valued new ideas and an open mind.

This is no longer the case.  Now I find myself constantly disagreeing with those who claim to be Agile.  I have come to associate the term “Agile” with a blinkered, prescriptive mindset.


Sorry Agile, I Need To Move On.

I’m sorry Agile, but it’s time for me to move on.

Moving On by Romy Mistra

I’m afraid you’re no longer the Agile I fell for.  Now you’re something else entirely.  I don’t like what you’ve become.

You used to have such high principles, but now all you want to do is set down rules.

Once when I looked at you I see alluring mystery.  You used to be so magical, so full of surprises and apparent contradictions.  Now all I see are to-do lists and time boxes.

We don’t like to listen to the same sounds.  I still adore the relational model.  Every time I listen I discover something new.  You just find it old fashioned.  You tell me I should be listening to NoSQL but it just sounds like a lot of noise to me.  I want to experiment and solve new problems but you seem satisfied with BDD.  Hearing those same three chords repeated over and over again is boring me.  I’m sick of you dismissing everything I find interesting as ‘too waterfall‘.

I hope we can stay friends.   It has been an interesting journey, but now I need to find a path less travelled.


Facts Not Opinions

The need for Experimentation

As aspiring Software Craftsmen we are always looking for way to raise the bar for professional software development. Practicing the crafting of high quality code that keeps adding value is essential. Working together in a community of professionals enables us to teach and learn from our shared experience.

However, if we really want to raise the bar of software craftsmanship, I believe we also need to perform experiments. Do you agree with me? If you don’t, perhaps it is because we have a different idea about what an experiment is.

If we look back in history we see the industrial revolution changing the world when a groups of craftsman came together to raise the bar for their various industries. Back then craftsmanship and engineering were the same thing. It is a time we have much to learn from them. Let’s consider one man, testing pioneer David Kirkaldy.

If you visit Kirkaldy’s Testing museum on Southwark High Street, just behind the Tate Modern, you’ll see David Kirkaldy’s motto carved above the door: “facts not options.” I think we have a lot to learn from David Kirkaldy. He performed experiments that replaced conflicting opinions with clear facts. He was made famous for his work related to the first Tay Bridge.

Bridge Building and Software Development

It is often said that writing software is not like building bridges, as Stack Overflow founders Jeff Atwood puts it: (original emphasis)

I find these discussions extremely frustrating, because I don’t think bridge building has anything in common with software development.* It’s a specious comparison. Software development is only like bridge building if you’re building a bridge on the planet Jupiter, out of newly invented materials, using construction equipment that didn’t exist five years ago.

http://www.codinghorror.com/blog/2005/05/bridges-software-engineering-and-god.html

We don’t have to go very far to find a time when bridge building was just like software development. It was just one hundred and fifty years ago that bridges were being built out of newly invented materials using construction equipment that didn’t exist a few years before.

When we think of timeless engineering we might envisage the Fowler and Baker’s Forth Bridge. Many might be surprised to discover that the famous design was not the first to be chosen. An earlier design by Thomas Bouch had been commissioned and the foundation had already been laid before the design was found to be inadequate.



Let us consider Thomas Bouch for a moment. He had an impressive reputation, having helped with the invention of train ferries and the construction of rail lines. His Tay Bridge had successfully passed three days of inspection to be declared safe for public traffic. In June 1879 Queen Victoria herself travelled across the bridge and knighted Bouch for his achievement. He appeared to have an impressive track record of successful projects despite some mishaps caused by poor engineering practices and sloppy shortcuts.

The general opinion was that Bouch was an excellent engineer. The facts, however, were quite different. In December 1879, the Tay Bridge collapsed with 60 lives lost. The official enquiry concluded that the bridge had been “badly designed, badly built and badly maintained, and that its downfall was due to inherent defects in the structure, which must sooner or later have brought it down.”

A badly designed bridge was poorly constructed and yet it managed to pass what appeared to be rigorous acceptance testing. Then it was poorly maintained before finally collapsing disastrously. Is it possible to imagine that bridge building back then was anything like software development is today?

So how did bridge building change? Is there anything that we can learn from the pioneers who brought about those changes? Can we follow in their footsteps so that the crafting of code might one day be held in the same regard as the building of bridges?

As Software Craftsmen we may be able to relate with surprising ease to David Kirkaldy and the way the Scottish engineer revolutionised testing with the invention of the Tensometer,

From Opinions to Facts

Alistair Cockburn describes the difference between bridge building and software development:

Civil engineers who design bridges are not supposed to invent new structures. Given a river and a projected traffic load, they are supposed to take soil samples and use code books to look for the simplest structure that handles the required load over the given distance, building on the soil at hand. They base their work on centuries of tabulation of known solutions.

Chapter 1 of Agile software development: the cooperative game

“Centuries” may be over stating the length of time these tabulations have been recorded for. For wrought-iron and steel it started with David Kirkaldy. When he started investigating the matter in 1862 he was surprised to discover very little was to be found:

It seems remarkable that whilst we have the results of many important and reliable experiments on Cast-iron, extremely few have been made, or at least published, on Wrought-iron, and almost none on Steel.

Results of an experimental inquiry into the comparitive tensile strength and other properties of verious kinds of wrought-iron and steel.

While very little hard experimental data was to be found on the subject, there was no shortage of opinions:

Although much has been written on the subject of wrought-iron and steel, yet, such is the great diversity of opinions held and stated by different individuals.

ibid

Kirkaldy’s solution to the problem was to provide experimental data.

It is hoped the results of these experiments, intended simply to elicit the truth, will be considered worthy of examination by those interested, and also at the same time prove of practical utility.

ibid

The same lack of facts and diversity of opinions has been observed in software In his book “Software Conflict” Robert L Glass observes:

In the processional literature we tend to see opinions presented as truth and advocacy presented as fact, with nothing acknowledging the tentative nature of some of these facts and much of this truth. Even noted computer scientist David Parnas has labeled much of our computer science truth “folklore,” because it has not been experimentally verified.

http://www.developerdotstar.com/books/software_conflict_glass.html

How can we progress past the diverse opinions of advocacy to a better understanding of the materials we work with? Is it only possible with large universities and generous grants? Is it within the practical reach of regular practitioners like ourselves?

Kirkaldy was a regular practitioner just like us. He built the tool needed and carried out his experiments as a personal side project:

At the time it was only intended to test a few specimens of each, but the investigation proved so interesting in itself and so likely to conduce to important practical results, that I was induced… to extend the experiments, as leisure and opportunity offered, very considerably beyond what had been originally contemplated.

Results of an experimental inquiry into the comparitive tensile strength and other properties of verious kinds of wrought-iron and steel.


His invention, the Tensometer, was the jUnit of its day, it wasn’t overly clever but it did it’s job well:

The apparatus employed was of the simplest construction, and proved during the experiment to work most satisfactory.

ibid

You can go and see it today, in the Kirkaldy’s Testing museum in London on Southwark High Street, just behind the Tate Modern. To our eyes it may look like a monster of a machine, but working at this scale was all in a day’s work for the Victorian engineer.

Is experimentation applicable to Software Craftsmanship? Certainly we do not work with iron and steel like the Victorian engineers, but that does not mean that experimentation is not applicable. These engineers were learning from the work of chemists who worked with liquid and gases. The materials were very different, but the principles of truth and rigour remained the same.

Practical vs Academic Experimentation

As craftsmen we are practical people. Do we really have time to experiment? Is it really a productive way to add value?

Today we think of experiments as the exclusive realm of the academic given to the pursuit of abstract goals, not the hard working practitioner. This was not always the case. For David Kirkaldy experimentation was essential if the conflicting complex theories and opinions were to be replaced with straightforward, simple facts.

The academics were there, too. They were usually wealthy members of the Royal Society. Being of independent means they were not bound by the need to earn a living. Sometimes there would be conflict between the two groups of experimenters. Consider, for example, the Safety Lamp, an important invention that was made, independently through careful experimentation, by two different people during the same year: 1815.


Sir Humphrey Davey invented the Davey Lamp. He was a knight of the realm, first baronet and a fellow of the royal society. He was already famous for his work on gases, such as the discovery of laughing gas (nitrous oxide). His lectures were well attended in fashionable London. No only did he invent a safety lamp, but he also progressed the scientific understanding of hydrogen.

It would be a couple decades before George Stephenson would become historically famous for building the first railways. He was an unknown engine-wright in the north of England responsible for maintaining and repairing the steam engines used at the collieries of Killingworth.

Stephenson was largely self educated and his experiments did not make him popular. However he made them with a practical purpose in mind. Samual Smiles relates:

For several years he had engaged, in his own rude way, in making experiments with the fire-damp in the Killingworth mine… One of the sinkers, observing him holding up lighted candles to the windward “blower” or fissure from which the inflammable gas escaped, entreated him to desist; but Stephenson’s answer was, that “he was busy with a plan which he hoped to make his experiments useful for preserving men’s lives.” On these occasions the miners usually got out of the way before he lit the gas.

Lives of the Engineeers


Stephenson’s work did prove useful. His theory was flawed and his risk management was poor: The experiments with his prototype lamps involved carrying it into a pit known to be full of explosive gasses. He did not advance scientific theory, but he did create a working safety lamp. Some controversy followed regarding who should take credit for the invention.

While Davey was clearly the better scientist, the question remains regarding whose experiments yielded the best results. In normal conditions both lamps performed equally well. However, in some exceptional circumstances there was a very important difference. Davey’s would burn red hot and potentially cause an explosion while Stephenson’s would safely go out:

A sudden outburst of gas took place… Upon this occasion, the whole of the Stephenson’s lamps, over a space of five hundred yards, were extinguished almost instantaneously; whereas the Davy lamps filled with fire and became red-hot… Had a strong current of air been blowing through the gallery at the time, an explosion would most probably taken place.

ibid

The lamps were put to rigorous testing by Dr Pereira for the Committee on Accidents in Mines. While both lamps had their faults, the conclusion was that “when exposed to a current of explosive gas the Davy lamp is ‘decidedly unsafe,’ and that the experiments by which its safety had been “demonstrated” in the lecture-room had proved entire ‘fallacious.'” On the ground practical experimentation results in better products, not inferior science. I know which lamp I would have preferred.

Software Experiments

Having established that the experiments are not only carried out by serious scientists in white lab coats but also sober mutton chopped engineers in frock coats, how about software developers in t-shirts and trainers? Does experimentation have any place when writing software?

Experimentation was once common practice among practitioners. In his book “Software Conflict” Robert L Glass relates the finding of an area of research called “protocol analysis” where observers would sit quietly and observe practitioners at work. They filmed them and then scrutinised the tapes to see how the design process worked. The process is familiar to all of us:

    • Understanding the problem
    • decomposing the problem into goals and objects
    • selecting and composing plans to solve the problem
    • implementing the plans
    • reflecting on the product and the process

http://www.developerdotstar.com/books/software_conflict_glass.html


It is the way in which these software engineers pursued the second step, the decomposition of the problem into goals and objects, that we might find surprising.

The designers, mentally and at lightening speed, were doing the following things:

    1. The constructed a mental model of a proposed solution to the problem.
    2. They mentally executed the model – in essence, running a simulation of the model – to see if it solved the problem.
    3. When they found that it didn’t (usually because it was too simple), they played the inadequate model back against those parts of the problem to see where it failed, and enhanced the model in those areas.
    4. They repeated steps 1-3 until they had a model that appeared to solve the problem.

http://www.developerdotstar.com/books/software_conflict_glass.html


Are you taking an Agile approach to your coding? If you are, then good for you. Now, are you following the approach described above? Are you constructing models, either in your head or as tests, and then testing them to destruction so that you can find where the model fails and improvement is required?

Or does your process more closely resemble the approach taken by those teams that were observed to fail:

These same researchers have explored the problems of people who are not very good at design. Those people tend to build representations of a design rather than models; they are then unable to perform simulation runs; and the result is they invent and are stuck with inadequate design solutions.

http://www.developerdotstar.com/books/software_conflict_glass.html


In Scrum I have observed the process where architects create complex, epic stories that span many months of work. These stories are a representation of the final system. They are broken down into smaller stories that are then fed to the developers in sprint size chunks. The developers are simply order takers, being fed their orders just in time. They have no opportunity to build and refine their own models through practical experimentation.

A recent thread on the Lean seems to indicate that I am not alone. John Herbert describes a similar situation.

Relegating developers to ‘order takers’ is exactly what I am describing. This is basically what the developers are saying, they have no room/time for creative thinking. That is at the root of my question. How do we achieve this balance? The dev team is on pretty strict 2 week sprint schedule, but it seems the 2 weeks has no time built in for anything other than getting the specific requirement developed. Where is the ‘alternate’ solutioning, or outside the box thinking achieved?? …

So where do we assign time for innovation without excluding any members of the team?

http://tech.groups.yahoo.com/group/leandevelopment/message/5920


Are you working on a tight sprint schedule working hard to deliver the backlog items promised? Are you in a position to say “Stop!” Can you declare the current approach a failure, and go back a few steps to pursue an alternative approach? Are such ideas unrealistic? Is is simply impractical? Then how come that was how successful programmers were able to work?

Our Software Craft

If we are Software Craftsmen then what is our craft? Is it the creation of code? No! If typing out lines of code is what we aspire to do how can we ever follow this principle of the Agile Manifesto?

Simplicity–the art of maximizing the amount of work not done–is essential.

http://agilemanifesto.org/principles.html

If the best code we will ever write is the code we avoid writing then how can writing code be our craft?

As Software Craftsmen we solve problems using code. To build those solutions we may use code, but first we must solve the problem.

Ask yourself: are you a Software Craftsman? If you are just taking orders rather than solving problems then you are not a craftsman, you are a factory worker. If you are learning, then is your mentor showing you how they solve the problem? Are they sharing their craft or simply giving orders?


Steam Punk Programming

I’ll be blogging about the Code Shares we’ve been running at the LJC and GDC soon.   In the meantime here’s my Steam Punk Programming lightening talk when I first announced the code share: http://skillsmatter.com/podcast/home/steam-punk-programming.


My Book Reviews on the LJC Book Club

I’m running the Book Club for the LJC and I’ve posted a couple of book reviews there:

I have a few more reviews in the pipeline for Cassandra, OSGi and Flow Based Programming.


Greg Wilson’s What We Actually Know About Software Development

There is an excellent talk from CUSEC given by Greg Wilson.  It’s called “What We Actually Know About Software Development, and Why We Believe It’s True.”

You can watch it here: http://vimeo.com/9270320

Here’s my notes on the talk.

The Lack of Evidence

11:10

Martin Fowler’s claims about Domain Specific Language

  • Using a DSL leads to:
    • Improved Productivity
    • Improved Communication
  • Although the article is an academic journal no citation or evidence is given.
  • A scotish verdict:
    • True
    • False
    • Not proven
  • Fowler claim’s that the debate is hampered because people don’t know how to do DSL’s properly.
  • Wilsno believes it is because of the low standards of proof.
  • We should have higher standards.
  • Things are getting better: there are more results from field studies.
  • Standards improve each year.

Estimation and Anchoring

14:50

Aranda & Easterbrook (2005): “Anchoring and Adjustment in Software Estimation”

Three groups are given a two page specification each. They are identical but for one paragraph. That one paragraph had a strong effect on the estimates provided.

Group Estimate Anchoring Paragraph
(lowball) 5.1 months “I admit I have no experience with software projects, but I guess this will take about 2 months to finish.”
Control 7.8 months “I’d like to give an estimate for this project myself, but I admit I have no experience estimating. We’ll wait for your calculation for an estimate.”
(highball) 15.4 months “I admit I have no experience with software projects, but I guess this will take about 20 months to finish.”

Rock Star Programmers and Poor Evidence Standards

18:18

“The best programmers are up to 28 times more productive than the worst”

  • Sackman, Erickson and Grant (1968) “Exploratory experimental studies comparing online and offline programming performance”
  • Study was designed to compare batch vs iterative approaches, not productivity.
  • Productivity measure was not explained.
  • Best vs worsts always exaggerates the effect. Standard deviation around the mean is better.
  • Just 12 programmers for an afternoon.
    • The next “major” study was 54 programmers for less than an hour.
  • In 1968 every programmer was self taught.

Improving Productivity

21:05

Look at the work of Lutz Prechelt

  • variations between programmers
  • effects of language
  • web programming framework

Studies are expensive and hard to do.

  • Not compared to the cost of drugs research.
  • A 5% productivity increase in a trillion dollar industry is worth a lot.

Two approaches of Pessimism and Optimism

23:40

Boem et al (1975) “Some Experience with Automated Aids to the Design of Large-Scale Reliable Software”

  1. Most errors are introduced during requirements analysis and design
  2. The later they are removed the more expensive it is to take them out.

Two approaches to this problem:

  • Pessimists
    • If we tackle the hump in the error injectino curve, fewer bugs will get to the expensive part of the fixing curve.”
  • Optimists
    • “If we do lots of short iterations, the total cost of fixing bugs will go down.”

Why are there so few women in software development?

25:55

Ceci & Williams (eds): Why Aren’t More Women in Science? Top Researchers Debate the Evidence.

There’s a review here: http://www.americanscientist.org/bookshelf/pub/changing-assumptions

28:22

Carol S Dwek’s essay (pdf) Is Math a Gift? Beliefs That Put Females at Risk 1)

  • Split the group into two groups.
    • Group One: “The tasks requires aptitude. ”
    • Group Two: “This task is entirely practice based.”
  • Group One does worse.
  • Even if the group are told that men have the aptitude more than women the men will still do worse.
  • When a difficulty is encountered the group members will conclude that they lack the aptitude and quit.

Improving Productivity (reprise)

30:58

  • For every 25% increase in problem complexity, there is a 100% increase in solution complexity. (Woodfield 1979)
    • Non-linear due to interaction effects.
    • Reducing problem complexity reduces the solution complexity.
    • Maybe this is why Agile works?
  • The two biggest causes of project failure are poor estimation and unstable requirements. (van Genuchten 1991 and many others)
    • There is no evidence that this is improving.
    • “I want you to make the web site more webish and not to webish. How long will that take?”
  • If more than 20-25% of a component has to be revised it’s better to rewrite it from scratch (Thomas et al, 1997)
    • Based on Flight Avionics. Very strict requirements.
    • Does it apply in other domains

Code Reviews

33:11

  • Rigorous inspections can remove 60-90% of errors before the first test is run. (Fagan 1975)
    • Code review is the best bug fixing method.
      • Better than unit tests.
      • Better than executing the code.
  • The first review and hour matter most. (Cohen 2006)
    • Having two people read the code is not economically effective.
    • How much code can you read in an hour?
    • A couple of hundred lines is as much as you can get through.
    • This supports the idea of making progress in little steps.
      • Open source projects reject large patches.

Conway’s Law

35:45

A system reflects the organisational structure that built it.

  • It was meant as a joke and turned out to be true. (Herbsleb et al 1999)
  • Physical distance doesn’t affect post-release fault rates but Distance in the organisational chart does.
    • Nagappan et all (2007) and Bird et al (2009)
    • Based on all the data from building Windows Vista. An enormous volume of data.
    • Searched for indicators of post release defect.
    • This goes against claims for the need for co-location.
    • Different managers with different goals has more impact than different continents.
  • Does this Explain why big companies produce such bad code.
    • Somebody from HP corporate headquarters: “We can’t just have people running around doing the right thing: there are rules!”

Scientific Progress

39:15

“Progress” sometimes means saying “Ooops.”

  • Science is different to religion because it accepts it’s mistakes.
  • For example, there appeared to be strong statistical evidence that cod metrics could predict post failure rates.
  • However, a later study showed that the co-relation was actually between code size and failure rates.
    • El Emam et al (2001): “The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics”
      • Most metrics’ values increase with code size.
      • If you do a double-barrelled correlation, the latter accounts for all the signal.
  • Sophisticated metrics are not needed: just a line count.
  • The results are disappointing, but raising the standards is a good thing.

Folk Medicine for Software

41:36

“Systematizing and synthesizing colloquial practice has been very productive in other disciplines.”

  • Look at what people are actually doing.
    • Take it back to the lab and investigate why it works.
  • The next decade of Software Engineering is looking at success and understanding why.

Beautiful Code

43:40

“What code is worth looking at, what code is worth reading?”

Beautiful Evidence

44:58

The Book Wilson was writing at the time. It’s now published: Making Software: What Really Works, and Why We Believe It

“What we know and why we think it’s true”

  • Knowledge transfer
  • A better textbook
  • Change the debate

A group of people will choose one thing they believe to be true about software development and they give the evidence as to why.

All text books for Software Development are crap.

  • Who really draws UML diagrams when thinking?

Software Craftsmanship

54:55

  • “It’s wondferful, but… show me the evidence”
  • Every ten years we need a new bandwagon so we can look innovative.

1) Wilson refers to it as Susan Dwek’s work
2) The name has already been stolen for a book by Edward Tuftle that is well wroth reading.


Follow

Get every new post delivered to your Inbox.