log - Fabian Linzbergers weblog

2010-03-24---klarna programming contest 2010

Wed, 24 Mar 2010 15:58:13 +0100

At the beginning of this week the results of the Klarna Programming Contest 2010 were published.

Most important news from my perspective:

Overall best solution:

Second prize: Fabian Linzberger (Apple iPhone)

Yay, I made second place! Yay, I won a smartphone!

Since I was asked about the problems I thought I would write up a bit of my experience.

Problem 1

(picture converted to png for the webbrowser. original PPM here)

A picture in PPM format (http://en.wikipedia.org/wiki/Netpbm_format#PPM_example), was given, that contained a hidden message.

I was totally mislead by the hint "what may seem red and insignificant is not". I thought it referred to the red text in the picture, when (obviously, in retrospect, haha!) it was in fact referring to the least significant bit of the red part of the pixels in the picture. I spent quite some time filtering the shades of grey around the text before I finally realized I was looking in the wrong place.

Basically some of the first 60 or so pixels would appear white to the eye just like the others around them, but in reality about half of them were just a tiny, tiny bit less red. Decode the white pixels to 0s and the less red pixels to 1s in bits, translate to letters and you had the secret message.

Problem 2

(picture converted to png for the webbrowser. original PPM here)

Another picture, another hidden message. Using the solution program from problem 1 gave a hint: "iccanobiF".

Clearly "Fibonacci", just backwards. Now it came in handy that I had actually programmed a decoder for the picture file for problem one. Searching around for suspicious pixel values, quickly revealed around 10 or so which were just a little less red, this time distributed somewhere among the blue of the tulip.

Turns out if you make a list of all blue and slightly less blue pixels in the picture, reverse it and use the Fibonacci sequence as indices you again get a list of bits that decode to letters. Reverse that once again and you had the second keyword. Phew.

Problem 3

A variant of the Game of Life. An encoding of letters to a start configurations of cells on a matrix and the other way round was specified. The solution program had to use this on a keyword, run a certain number of iterations and afterwards decode the resulting configuration of cells to another keyword.

This was a bit of work to implement, but overall I found it a lot easier than the previous two picture puzzles: at least the objective was clear.

Problem 4

A hex encoded and Vigenère encrypted message was given. The task was to decode it. The encryption key could be obtained by running another simulation on the solution program for problem 3. Again a bit of coding, but more straightforward than 1 and 2.

All my coding was done in Haskell. Overall experience was quiet pleasant. Converting bits to text felt like taking a bit too much effort, maybe because I am not used to the libraries. As always: Quickcheck rocks!

2010-02-28---tronbot-google ai challenge

Tue, 02 Mar 2010 10:01:13 +0100

I participated in the google ai challenge. In the end I was placed 134 out of 708: My profile url.

Since I am working on a Monte Carlo tree search based Computer Go program, I figured I would try to use that as a basis for my bot.

In general it worked quiet ok. Especially in close range combat it is often able to play quite well:

At longer range and when looking for the longest path in endgame situations lack of depth in the tree search shows and quiet many mistakes are made.

I also experimented with using the distance between the players as a heuristic to bias the tree search into deeper branches, this was only mildly successful however. It seemed to be very hard to strike a balance between the heuristic not making a difference and turning my bot into a chaser.

In the final version my bots strategy is as follows:

Test if there is a path reaching the opponent using the a* pathfinding algorithm.

#         ###############
        ##        ...   #
      ##          .#1   #
    ##      #     .###  #
   #        #     .     #
  #         #     .     #
 #          # ..... #   #
#           # .####     #
# ###       # .         #
#     ####  # .         #
#           # .  #      #
#     #       .  #      #
#     #    ###.  #      #
#     #    ....  #      #
#     #    .#           #
#         ..#  ###      #
#         . #      ###  #
#     ####. #           #
#   #  .... #          # 
#     ..    #         #  
#     .     #        #   
#  ###.     #      ##    
#   2#.          ##      
#   ...        ##        
###############

a* found shortest path between players

In case there is not, enter special endgame mode: the value of nodes in tree search is number of moves divided by flood fill size. I am unhappy with the endgame, this should make less mistakes.

In case opponents can still reach each other, use a biased tree search:

p and P are nodes searched for player 1
e and E are nodes searched for player 2 (enemy)
1 is the principal variation for player 1
1 is the principal variation for player 2

###############
#1PPPPPPP     #
#111PPPPP     #
#1111PPP     e#
#PPP1PP     eE#
#PPPPPPP   eEE#
#PPPPPPPe eEEE#
#PPPPPPEEEeEEE#
#PPP p EEEEEEE#
#PPp   EEEEEEE#
#Pp    EEE2EEE#
#      EEE2222#
#     EEEEEE22#
#    EEEEEEE22#
###############
nodes evaluated for each player in tree search at long range

At long range nodes closing in to the opponent are preferred.

Nodes are evaluated using flood fill if they divide opponents into separate areas.

Otherwise random playouts until either player crashes are used.

###############
##     PPP  e #
#### pPPPPWEEE#
#  #PPPP111222#
#  #PPP11W22E2#
#  #PPP11W2EEE#
#  #####1W2EEE#
#    ##1122EEE#
#    PPPW#EEE #
#     PP ##E  #
#         #   #
#         #####
#          ####
#          ####
###############
nodes evaluated for each player in tree search at closer range

Tree search terminates as soon as 0.9 seconds are spent and return the best move found so far.

After reading a little about strategies other people have used, it may also have been a bad heuristic to just approach the other bot. "Voronoi territory" - the number of points that can be reached by each bot first - may have been a better choice. In general a good heuristic is key for Monte Carlo tree search to be able to deeply explore the tree without missing the wrong branches, so this could have helped a lot.

Further links:

2010-01-07---sudoku

Thu, 07 Jan 2010 14:49:17 +0100

Lefants haskell sudoku solver

Sometime in autumn 2009 i created a sudoku solver in haskell as a programming exercise for myself. I am pretty happy with the result, it took no more than an afternoon to implement and has been able to solve everything I tried in a couple of seconds. Now I can happily smile to myself whenever I see someone solving sudokus from the newspaper

Below is a commented and syntax highlighted version of the main module, the complete code in git is also online.

If you are interested in more haskell solutions to the sudoku problem, make sure to check out the Sudoku page on haskellwiki

-- Lefants haskell sudoku solver
-- -----------------------------

{-# OPTIONS -O2 -Wall -Werror -Wwarn #-}

-- This is the main module, containing the actual logic.

module Sudoku (
               solveOne,
              ) where
import Data.List


{-
There are two helpers:

 * sudoku-test runs some (very very basic) tests.
 * sudoku-run is the binary for normal invocation, it will read from
   stdin and output to stdout. use it like this:

$ cat <<EOF | ./sudoku-run
.98......
....7....
....15...
1........
...2....9
...9.6.82
.......3.
5.1......
...4...2.
EOF

798624315
315879246
264315978
129587463
683241759
457936182
942158637
531762894
876493521
-}




{-
The Coord type is a three-dimensional coordinate, the 3rd one is the
box the field is in, like indicated here:

+-----------+
|111|222|333|
|111|222|333|
|111|222|333|
+-----------+
|444|555|666|
|444|555|666|
|444|555|666|
+-----------+
|777|888|999|
|777|888|999|
|777|888|999|
+-----------+
-}

type Coord = (Int, Int, Int)



-- Value holds a solution value or a list of remaining valid
-- candidates for the field.

data Value = Element Int | Options [Int]
           deriving (Show)

-- An actual field consists of a coordinate and a Value (as described
-- above).

type Pair = (Coord, Value)



-- This is the main exported function. It will read in a string of
-- digits or . and feed it to the solve' function which will find a
-- solution using solve and then return a prettified string
-- representation.

solveOne :: String -> String
solveOne ls =
    concatMap pretty $
    sortBy compareC $
    solve' $
    zip triples $
    map readOne ls


-- This will return a list of three-dimensional coordinates as
-- explained with the Coord type above.

triples :: [Coord]
triples = 
    zip3 a b $ map z pairs
    where
      pairs = [(a', b') | b' <- [1..9], a' <- [1..9]]
      (a, b) = unzip pairs

      z :: (Int, Int) -> Int
      z (x, y) =
          x2z + y2z
          where
            x2z = ((x - 1) `div` 3) + 1
            y2z = ((y - 1) `div` 3) * 3


-- Pretty representation of field values

pretty :: (t, Value) -> String
pretty (_, Element e) = show e
pretty (_, Options _) = ""


-- Used for sorting coordinates from left to right and top to bottom.

compareC :: (Ord t2, Ord t3) =>
            ((t2, t3, t4), t) -> ((t2, t3, t5), t1) -> Ordering
compareC (c1, _) (c2, _) =
    compareT c1 c2
    where
      compareT (a1, b1, _) (a2, b2, _)
          | b1 == b2  = compare a1 a2
          | otherwise = compare b1 b2



-- Read in a predefined single value or failing that initialize the
-- list of options.

readOne :: Char -> Value
readOne c =
    case c `elem` (map (head.show) ([1..9] :: [Int])) of
      True -> Element (read [c])
      False -> Options [1..9]



-- solve' and solve contain the actual solving logic. solve' will
-- partition the initial list of fields into ones containing single
-- elements (already defined / solved) and those containing a list of
-- remaining options.

solve' :: [Pair] -> [Pair]
solve' ls =
    solution
    where
      Just solution = solve done todo
      (done, todo) = partition isElement ls
      isElement :: (t, Value) -> Bool
      isElement (_, Element _) = True
      isElement (_, Options _) = False


-- solve takes two lists of coordinate / value pairs as parameters:
-- the first one contains solved single element fields, the second all
-- the lists with remaining options.

solve :: [Pair] -> [Pair] -> Maybe [Pair]
-- if all the fields have one element we are done.
solve es [] = Just es
solve es os =
    case as of
      -- no more Options, no solutions possible
      [] -> Nothing
      -- try first option
      (a : as') ->
          -- recurse using backtracking, if we can solve it
          case solve ((c, Element a) : es) os' of
            -- we are done
            Just es' ->
                Just es'
            -- this branch contains no solutions, retry without it
            Nothing ->
                solve es ((c, Options as') : os')

    where
      -- first prune all Options list at the current level, then order
      -- branches with *few* options first
      ((c, Options as) : os') = sortBy lessOptions $ map revaluate os

      lessOptions (_, Options xs) (_, Options ys) =
          compare (length xs) (length ys)
      lessOptions (_, Element _) _ =
          error "illegal lessOptions call"
      lessOptions (_, Options _) (_, Element _) =
          error "illegal lessOptions call"

      -- filter out other Options from list that are made impossible
      -- by choosing a certain one
      revaluate :: Pair -> Pair
      revaluate (c'@(x, y, z), Options aas) =
          {-# SCC "revaluate" #-}
          (c', Options aas')
          where
            aas' = aas \\ otherValues
            otherValues = map (\(_, Element e) -> e)
                          ((filter (\e -> x == px e) es) ++ 
                           (filter (\e -> y == py e) es) ++
                           (filter (\e -> z == pz e) es))
      revaluate ((_, _, _), Element _) =
          error "illegal revaluate call"


-- helper functions to project a single coordinate from a Pair

px :: Pair -> Int
px ((x, _, _), _) = x
py :: Pair -> Int
py ((_, y, _), _) = y
pz :: Pair -> Int
pz ((_, _, z), _) = z

2010-01-07---laby-programming-intro

Thu, 07 Jan 2010 11:27:15 +0100

The other day I was playing a little bit with laby, a small program to help in understanding basic programming concepts for kids, but maybe grown ups too.

It is available in Ubuntu in the laby package from karmic (9.10) on. I can highly recommend it, it is always fun to watch a graphical representation of a programmed creature do its job after the coding. I have created a general solution in ocaml that will successfully solve at least all the levels included in 0.5.0.

(* ocaml wall hugger for laby *)
let ant =
  (* go straight ahead til we hit something *)
  while (Robot.look () = `Void) do
    Robot.forward ();
  done;

  (* main wall hugin' loop *)
  while not (Robot.look () = `Exit) do

    if (Robot.look () == `Void)
    then (
      Robot.forward ();
      Robot.right ();
    );

    if (Robot.look () == `Rock)
    then (
      Robot.take ();
      Robot.forward ();
      Robot.left ();
      Robot.left ();
      Robot.drop ();
      Robot.left ();
    );

    if (Robot.look () == `Wall) or (Robot.look () == `Web)
    then
      Robot.left ();

  done;

  Robot.door_open ();
;;

download source file: laby.ml

Go little robot ant!

2007-11-25---some-backports

Fri, 20 Nov 2009 17:43:42 +0100

i have created debian backports to etch of current versions of mpd, rtorrent and mt-daapd and the dependencies needed to build / install them.

add

deb http://lefant.net/debian/ etch-backports main

to your /etc/apt/sources.list in case you are interested.

mt-daapd and rhythmbox are a great pair to store music on one computer and play on another, with little configuration on the server and none on the client, by the magic of zeroconf! highly recommended. should also work with itunes and other stuff from apple.

2007-10-11---ipv6 at debienna

Fri, 20 Nov 2009 17:43:42 +0100

yesterday at debienna i gave a talk about computer networking in general, ipv6 and vpn in particular. afterwards we had some nice further discussion.

i have also created a page in the wiki with some links in case you are interested: NetzwerkenConnectYourDevices

2007-11-25---voip-hacking-at-deepsec

Fri, 20 Nov 2009 17:43:42 +0100

tuesday and wednesday i attended a training titled "Practical VOIP/SIP Hacking" at the premiere of the future annual security conference deepsec in vienna. official site. the organisation was pretty professional, as to be expected for the business oriented pricing. the hotels network occassionally was a bit laggy, but that's probably not too surprising considering the demands of a bunch of workshops full of hackers doing network security stuff.

i really enjoyed it a lot. klaus really knows his sip and he addressed quite a range of typical configuration problems, possible exploits and how to fix them.

small hint for starters: never put openser on the internet in its default config, unless you want people to make free calls - you may be the one to pay the bill. other software of course mostly isn't much better, but default openser is literally configured as an open relay. cisco gateways are not much better, but since they are so expensive, will only ever be operated by real experts - haha

an excellent resource if you are interested in this field is also klaus's voip link collection.

i was definetely able to profit a lot for the system i am designing at work. stay tuned for some config samples as it becomes more and more production proofed.

2007-08-23---debienna

Fri, 20 Nov 2009 17:43:42 +0100

highlights and random thoughts:

present at some point in the evening or another: lefant, harald, karl, andreas, maks, rotty, markus, gregor

i play around with the new metalab 2.0 soup.io, there is now soup.lefant.net. also there is a aggregate plugin of ikiwiki resulting in lefants soup on ikiwiki.

looking at the quintessenz backup server in the office i find that backuppc is slow and boxbackup is quick and efficient. i also run a boxbackup server at home on my nslu2 (32mb ram for those who don't know it), it is also in debian sid and there is an etch version on backports.org.

it seems an official darcs.debian.org is being planned. i was contacted as the debian maintainer for darcsweb and hurried to finally reply to two of the open bugs. of course i hope darcs.debian.org will feature darcsweb

checking out firehol. default config works good enough for my laptop

after about a week of use rss2imap seems to be good enough as a google reader replacement. a little of my privacy returns

http://www.mozart-oz.org/features.html is surely interesting for the functionally inclined.

at a later time we also discussed some further topics we would like to talk about in the future.

cheap voip at voipcheap

Fri, 20 Nov 2009 17:43:42 +0100

2007-06-02

since i am implementing a voice over ip project at work, i naturally am also investigating options at home. for now i am running an asterisk server on my webserver and use it to call out to PSTN (mostly my girlfriend to her brother who is doing his civil service in sierra leone). fellow debian hacker klaus ita recommended a call by call provider: voipcheap

they seem to live up to their name and offer really nice rates. also austrian land lines are free as in free beer. you have to download a windows client to complete the registration which really sucks, but after delaying for more than two weeks i finally got around to do it today. 5 minutes later i am successfully placing my first free call to an austrian landline after adding something like the following to my asterisks sip.conf:

[voipcheap]
realm=sip.voipcheap.com
host=sip.voipcheap.com
username=<myname>
secret= <mypassword>
type=peer
qualify=no

(instructions found on here)

2007-07-22---phpfspot

Fri, 20 Nov 2009 17:43:42 +0100

2007-07-22

for some time now i have been using f-spot (website: http://f-spot.org/, which is a nice photo organization software for gnome. above all f-spot has great support for tagging, which in my opinion is a very important feature. you can also automatically create a static html thumbnail gallery for your website or export to flickr or gallery (the php thing many people seem to be using). however this way either you loose your tags (static html, gallery) or you handover your pictures and control over the url to flickr (which limits you to 200 pictures in their free plan anyway).

phpfspot (website: http://oss.netshadow.at/oss/phpfspot/Project, freshmeat: http://freshmeat.net/projects/phpfspot/) to the rescue!

after some whining on my part at work my colleague at work andreas unterkircher started looking for a better webfrontend solution (foremost including good support for the tags) himself and, finding none, started coding. for reference there is pennave with a similar goal (website: [http://pennave.sourceforge.net/](http://pennave.sourceforge. net/)), however speed was really lacking and development very slow).

i am now happily running phpfspot myself (link: http://gallery.lefant.net/). i also use the thumbnail export feature to create thumbnail galleries for use in the debienna wiki (link: http://debienna.at/). the thumbnails on the starting page are created dynamically for each page visit (try reloading, see http://debienna.at/?action=raw for the page source and the ?RandomQuote hack) which uses the FotoFortunes wiki page as its source (warning, huge page). using phpfspots export feature i can now create the content of the ?FotoFortunes page automatically after selecting all pictures with the debienna tag.

the same works with literal html for blog entries like this one. find a thumbnail gallery of all my images tagged with "screens" below:

i am happy, thanks unki!