Sunday, July 17, 2011

Trial and Error and the God Complex...

I just saw Tim Harford's TED talk: Trial and Error and the God Complex.

Harford makes some good arguments for evolutionary, "trial and error" approaches to solving complex problems. There are also significant risks.

Harford's examples are very carefully chosen not to present the limitations of evolutionary approaches to solve problems. For example, a human baby is a product of evolution, but without a specific design -- a product of natural environmental constraint and common decent. Harford's "nozzle" story is different, there is a desired design, which is found through a genetic algorithm... in this case, the principle of evolution is applied in an artificial environment where the constraint is a metric on a fitness function (e.g. how well does the nozzle make detergent?) This allows the genetic algorithm to incrementally improve the design without actually knowing how the design works, by passing traits that contribute to passing the fitness function.

It is helpful to realize that top-down design and genetic algorithm design are simply different ways of finding solutions (optima). One isn't better than the other, any more than analytic solutions to functions are better than numerical solutions, nor are they always equivalently easy or possible to find.

Genetic algorithms only work for designs when a fitness metric can be established (e.g. how well the nozzle makes detergent) -- it is not useful or appropriate as a technique when a fitness function is unknown. Hence, you already have to have expended considerable thought to define a fitness function that correctly specifies the question in order to solve problems the way Harford suggests. This isn't trivial in the general case, it can be harder than solving the problem using top-down design methods.

Harford's example of the stock market is not completely unrealistic -- it is entirely likely that "something" is evolving faster than humans -- but since we have only very vague ideas about what effects a fitness function like "fiduciary trust" has, we don't know exactly what corporations will evolve to claim us as pets.

This brings us to the second shortcoming of evolutionary techniques.... while they are excellent at homing in on optima using a fitness function, the only thing you can say about them is the probability that they are good enough to pass the fitness function and nothing more. If your fitness function forgets to include exceptions, there is no guarantee that your solution will remain stable for such conditions. Even more fundamentally, you have no idea how the solution actually works.

Genetic algorithms can lead scientists to breakthroughs, however in order to understand how they work, the solutions must be reverse-engineered back into an analytic understanding.

Harford focuses so closely on the amazing ability of these systems to find answers that he has forgotten that those answers need rigorous testing. It turns out that humans are even worse at defining correct testing scenarios than we are at building systems top-down. So while his approach may be compelling, it is not without significant danger. It substitutes one evil (i.e. what he calls the "God complex") with another: the hubris to think that we know what we want (i.e. understand the fitness function) sufficiently to know when we get it.

Asimov warned us of this hubris in stories featuring the notorious well-meaning "Laws of Robotics" that led robots to very different conclusions than expected. And Crichton warned us of trying to control such systems as they rapidly change and exceed our ability to control them.

Real evolution is raw, untamed, and unpredictable -- it doesn't play by rules we fully comprehend.

Monday, July 11, 2011

Good Lesson, but not universal...

Aaron Iba explains in his blog that his iPhone game led him to a new way of thinking about development that he thinks makes him a better programmer and startup founder.

"Iterate-and-Repair" is a good optimization strategy where opportunity costs are low (i.e. trying out different solutions in a puzzle game), but they aren't so great where opportunity costs are high (i.e. trying to send a man to the moon). In that case, the best strategy is still to engineer an optimal solution using mathematics.

The key distinction is not learning styles, nor even an MIT background, but rather asking the question: "what is the opportunity cost for my solution?"

The late 90's saw a wave of "good enough" engineering that was based on a similar idea of iterate-and-repair, but notoriously did not factor opportunity cost in the analysis. Who needs all those edge-cases and testing when you can follow the 80-20 pareto rule and get to market quickly? Well, if you've ignored the opportunity costs and your solution fries a thousand people in an aircraft, or tanks a billion dollars in stocks overnight, you better believe that an 80/20 approach is not "good enough" after all.

Of course you should use the right tool for the job, but there is still a need for engineered solutions.

Thursday, February 17, 2011

ruby and json vs xml marshalling

I've been watching a silent war rage between the ruby community with its preference for JSON and RESTful services and the Java community with its preference for XML and SOAP.

The rdoc for Nokogiri snarks at the Java community with a simple quote:

"XML is like violence - if it doesn’t solve your problems, you are not using enough of it."

Like many devs caught in the middle of this war (namely having to integrate Rails apps with Java SOA backends) I'm never quite satisfied with the state of Ruby XML marshaling. Sure there are a dozen gems to address this -- they always vary between giant thousand-object DOMs that leak memory like a sieve, arcane forced DSL syntaxes, or simple to use, but incomplete APIs. So I started to think about hand rolling "yet another XML marshaller" for Ruby.


My first idea was to simplify the XML DOM, so that everything is unmarshaled as an element. Then I can use a simple hierarchal accessor, the "dot" operator on the Ruby object side:




On the face of it, I thought that looked pretty sweet. But then I considered remarshaling this object as XML. Doh! There's no way to tell whether "mike" is an element or an attribute... well I could stick a little metadata on there to remind me later, but it's starting to look like I stumbled into the same path that so many other Ruby gems had trying to solve this problem. And it's not for lack of trying -- Ruby devs have tried many ways to resolve this ambiguity with various results, but not a lot of clear wins -- there's always a "gotcha" in there that makes you wince when you have to handle it. (it's the same wince that Java devs have when they switch axis jars on one server but not another.)


I sat back in disgust and thought to myself,

"wow, this would be so much easier if it was just JSON..."

then it hit me... JSON is easy because round-trip marshaling doesn't introduce any ambiguity about elements vs attributes! Everything is an element!!!

No wonder JSON is so easy!

Saturday, February 5, 2011

How to get Surface 1.0 SP1 running on Win7 x64 and VS2010

I'm posting this "brain dump" from my research journal because it was a royal PITA. Maybe it can help someone else. The articles referenced in here hopscotch around in quite a tangled fashion. I empathize with the people who wrote these up... we all have the tendency to document the things we know about at the time we know them and then quickly go stale or have quirks. This post will be no different, but at least as of 2/5/2011, this tells devs new to Microsoft Surface 1.0 SP1 development how to proceed. I hope it can save some grief.

Twitter thanks to @GrumpyDev, @eNeRGy164, @surface, @sevensteps, @brianpeek, my friend @gaganrajpal, and Prof Holly Yango for her quickstart (http://www.cs.uml.edu/~holly/)



back to work. going to try to install the SDK again and get it working.
previously I had used Brian Peek’s instructions at http://www.brianpeek.com/blog/archive/2009/03/10/install-the-surface-sdk-on-windows-7-and-or-x64.aspx but there are some slight variations here. @sevensteps advised that grumpydev was worked.

BULLETS mark steps I did.

from my 1/27 post:
GrumpyDev was identified as should work if I follow it to the letter, will try this weekend again: http://www.grumpydev.com/2009/05/17/surface-sdk-sp1-on-vista-and-win7-x64/
  • use ORCA to remove the following keys from the MSI.
  • Select “LaunchCondition” in the left hand list, then select “Install OR NOT VersionNT64″. delete
  • select “InstallExecuteSequence” in the list on the left and remove SetCreateSqmMachineGuid and CreateSqmMachineGuid. delete

Followed this to the right after it says save and close ORCA part, but before installing flip over to these instructions to get orca edits for VS2010 as well:
http://blog.hompus.nl/2010/04/14/using-the-surface-sdk-with-visual-studio-2010/
  • Select “LaunchCondition” and:
  • Select the row with "Installed OR (VS2008SPLEVEL AND VS2008CSPROJSUPPORT) OR VCSEXP2008SPLEVEL" and choose "Drop row"
  • Select the row with "Installed OR (VS2008SPLEVEL AND VS2008SPLEVEL >= "#0") OR (VCSEXP2008SPLEVEL AND VCSEXP2008SPLEVEL >= "#0")" and choose "Drop row"
  • Select the row with "Installed OR DEXPLORE" and choose "Drop row"
  • Select the row with "Installed OR VS90DEVENV OR NOT VS2008SPLEVEL" and choose "Drop row"
  • Select the row with "Installed OR VCSHARP90EXPRESS OR NOT VCSEXP2008SPLEVEL" and choose "Drop row"

Now, continue from [http://www.grumpydev.com/2009/05/17/surface-sdk-sp1-on-vista-and-win7-x64/] and:
  • Now you can quit Orca and run the msi, which should install just fine. Make sure you also switch off automatic updates and error reporting as follows...
  • done

Ok, the SDK is installed. Now, following the instructions it says to go to the original post and follow steps 2 & 3 (http://www.grumpydev.com/2008/12/26/surface-sdk-on-vista-x64/):

me:
  • changed step 1 to “open command prompt with Run as Admin”
    • cd “C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC”
    • vcvarsall.bat
    • then:
  • Change to "C:\Program Files (x86)\Microsoft SDKs\Surface\v1.0\Tools\Simulator".
  • Enter "corflags SurfaceSimulator.exe /32bit+ /force" (without the quotes). We need to use /force because the assemblies are strong named signed, and altering them this way will invalidate that signature. This is unfortunately unavoidable, but I haven’t found any problems from doing so. You will receive a CF011 warning informing you of this
  • Change to "C:\Program Files (x86)\Microsoft Surface\v1.0".
  • Use the same corflags syntax from step 3 on the following files, one by one:Attract.exe, AttractConfig.exe, SurfaceInput.exe, SurfaceOutOfOrder.exe,SurfaceShell.exe.
  • Close the command prompt and launch the simulator!

finished step 2. simulator appears to run. however closing it doesn’t shut it down. I had to go to the task explorer and kill the process tree for the simulator. still, this is a step in the right direction.

Now, on to step 3. but wait, I don’t have any projects yet. Ok, we’ll come back to this.

Going back to http://www.grumpydev.com/2009/05/17/surface-sdk-sp1-on-vista-and-win7-x64/ he explains in an update how to install the samples.

I had to do this:
  • goto C:\Program Files (x86)\Microsoft SDKs\Surface\v1.0\Samples and unzip “Surface Code Samples.zip” into the same dir.
    • had to expand this to a different folder.
    • now follow his update:
  • Edit the configuration properties for the sample solution and create an x86 platform for all projects. (this is step3, haven’t done it yet)
  • - Edit InstallSamples.bat
  • - Comment out the lines on either side of the set as follows:
  • ::FOR /F “eol=H tokens=2*” %%A IN (‘REG QUERY HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Surface\v1.0 /v IsLogicalSurfaceUnit’) DO SET LogicalTableValue=%%B
  • SET InstallingOnTable=false
  • ::IF %LogicalTableValue%==0×1 SET InstallingOnTable=true
  • - Change the MSBuildParameters to reference the x86 PlatformName as follows:
  • SET MSBuildParameters=/p:Configuration=Release;PlatformName=x86 /noconsolelogger /fl /fileLoggerParameters:LogFile=%LogFile%;Append /nologo


ok, now I did:
  • run cmd as admin
    • cd “C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC”
    • vcvarsall.bat
  • cd (where I unzipped the files)
  • InstallSamples.bat


“Successfully installed...”
“Please run the Surface Simulator to browse sample applications from the Surface Shell Application launcher”

Hmmm, wonder what that is? oh!! just running the simulator now has a bunch of apps in it. Ok, there must be a directory or database where the samples got “installed” to in order to run in the simulator...

Hmm however none of the samples actually run when I open them -- timeout. Hmmm.

Wait, maybe the XNA stuff isn’t installed?
Reading this: http://blog.hompus.nl/2010/03/03/installing-the-microsoft-surface-sdk-on-windows-7-x64/

Nope, I had already done this, tried uninstalling and reinstalling. no dice.

Ok, what about this? I noticed that grumpydev http://www.grumpydev.com/2008/12/26/surface-sdk-on-vista-x64/ step 2, substep 5 was missing this file:
setupcustomaction.exe
this is mentioned at http://blog.hompus.nl/2010/03/03/installing-the-microsoft-surface-sdk-on-windows-7-x64/
so I’m doing this from that dir:
  • CorFlags setupcustomaction.exe /32BIT+ /Force /nologo

Still no dice... everytime I launch a sample, it starts up, I click something and get “ControlsBox has stopped working” -- or similar depending on the sample.


Ok, forget the samples. Let’s go back to the instructions for installing the solution templates:
http://blog.hompus.nl/2010/04/14/using-the-surface-sdk-with-visual-studio-2010/

COPYING THE TEMPLATES FROM THE SDK:
xcopy /s "C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\ItemTemplates\CSharp\Surface\v1.0" "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\ItemTemplates\CSharp\Surface\1033\"
xcopy /s "C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\ProjectTemplates\CSharp\Surface\v1.0" "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\ProjectTemplates\CSharp\Surface\1033\"

ok, that was bunk. those dirs don’t exist on my system. I think he meant:

C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\VCSExpress\ItemTemplates\CSharp\Surface\v1.0

WpfSurfaceCustomControl.zip

WpfSurfaceUserControl.zip

WpfSurfaceWindow.zip

WpfTagVisualization.zip


which did exist... but hmmm, there is also...

C:\Program Files (x86)\Microsoft SDKs\Surface\v1.0\Project Templates

WpfSurfaceApplication.zip

XnaGS3SurfaceApplication.zip

XnaSurfaceApplication.zip


SO CONFUSING!!

Wait, hold on, the target directory already exists too!!:

C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\ProjectTemplates\CSharp\Surface\1033

WpfSurfaceApplication.zip

XnaSurfaceApplication.zip


each of these dirs has different files. GOOD GRIEF!!

Ok, let’s just try running the last two commands and see if VS2010 recognizes the two templates installed in VS 10.0 dir above.
  • cd "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE"
  • devenv /setup


Ok, after it completed it picked up the two installed templates in the VS2010. Good! To see this:
  • Open VS2010 and go to “New Project”
  • expand “Other Languages/Visual C#/Surface”. See:
    • “Surface Application (WPF)”
    • “Surface Application (XNA)”



NOTE: http://msdn.microsoft.com/en-us/library/ee804897(v=surface.10).aspx says:
The Microsoft Windows SDK for Windows Server 2008 and .NET Framework 3.5 is not required. However, if you want to develop Microsoft Surface applications by using Windows APIs, you must install the Windows SDK for Windows Server 2008 and .NET Framework 3.5.
(I already had installed Windows SDK (not server 2008) and .NET 3.5 previously).




ALSO NOTE from the same loc:
The Microsoft Surface SDK does include project templates for XNA 3.0, but it does not include XNA 3.0 components. If you want to develop Microsoft Surface applications by using XNA 3.0, install Microsoft XNA Game Studio 3.0. When you want to deploy a Microsoft Surface application that uses XNA 3.0, install Microsoft XNA Framework Redistributable 3.0 on all Microsoft Surface units that will run the application.
(I had installed the XNA Game Studio 3.0 previously).



Note that http://blog.hompus.nl/2010/04/14/using-the-surface-sdk-with-visual-studio-2010/ also states you MUST select .NET 3.5 when creating the project from the template.
  • make sure .NET Framework 3.5 is selected at the top
  • I want the XNA template, so I click that, fillin the name, location and OK.


Horray, now I have a solution template filled in. now I can go back to step 3 of http://www.grumpydev.com/2008/12/26/surface-sdk-on-vista-x64/

I had to adapt these directions...
  • right click the solution and select “Configuration Manager...”
  • set “Active Solution configuration” to Debug
    • from “Active solution platform” select “New...”
    • select “x86” and OK.
  • make sure that “Platform” in the list below is set to x86.
  • set “Active Solution configuration” to Release
  • select x86.
  • make sure that “Platform” in the list below is set to x86.
  • close.


Now, I wonder if we can get this to run in the surface simulator?
http://msdn.microsoft.com/en-us/library/ee804880(v=surface.10).aspx
  • run the surfacesimulator first
  • then hit debug in the project.


Ok, that seemed to launch the blank XNA template in the simulator... so far, so good.

Let’s see if something works... found this quick start:
http://www.cs.uml.edu/~holly/teaching/91550/fall2009/handouts/lab2/XNA%20Quick%20Start.pdf

changing the sprite to “Resources\icon.png” (which already existed in the template) worked... I get the icon in the middle of the display.

WOOT!

Ok, next up, time to dig into XNA and see about graphics, and shaders, etc. I may start with simple particle sims.

Wednesday, January 5, 2011

patch straight from github!

Wow, this is a cool trick...

1) go to any code on github
2) click on the "commit" link for any changeset
3) add ".diff" to the url
4) now copy the url and voila:

curl [url] | patch -d [dir]

e.g.

curl http://github.com/rails/rails/commit/268c9040d5c3c7ed30f3923eee71a78eeece8a8a.diff | sudo patch -d /Library/Ruby/Gems/1.8/gems/rails-2.3.5/lib/rails


So easy!

(courtesy of http://www.mattvsworld.com/blog/2010/03/version_requirements-deprecated-warning-in-rails/ and http://www.kartar.net/2009/09/output-github-commits-as-unified-diffs/)