Skip navigation

I’ve been using git for version control for a few years now and at several different companies (although I still use subversion for personal projects).  I think I finally get it and understand the common workflow with it so I wanted to write it down.  This largely involves the rebase command (a command which I was initially weary of), hence the post title.

For starters, unlike many people these days, I don’t think git is absolutely superior to other version control systems.  As I said, I still use subversion for my personal stuff.  So let’s start with some negatives.

First, git doesn’t store directories or history about them.  So if you you want an empty directory, you need to add a hidden file (.gitignore is a good choice) to the “empty” directory.  As someone who moved from cvs to subversion, this feels like a step backwards.  At least in cvs, I understand that it is an unavoidable artifact of using rcs files, but with git I really don’t understand the decision.

Second, git is built very much around the idea of a single project per repository.  There are situations where this makes a lot of sense, but there are use cases where I would prefer to have multiple things in one repository and just check out the parts you want.

Finally, I miss cvs tags in files (which subversion supports just fine).

I’m in a good mood so I won’t count the documentation against git.

One of the things people always say is that git is great because it is decentralized and I’m not sure I buy that.  In every place I’ve worked (and every major project using it that I’ve looked at), there is always one master, central, blessed repository that everyone else clones and pushes to (sometimes indirectly if only limited people have access).  I think this makes sense, but it makes the decentralized talk seem a little funny.

The other part of it being decentralized is that you have a complete backup of the repo with every clone.  Except that git clone only clones some things.  You can fix this (more or less) with:
$ git fetch --all --tags

Now that I’m done with all that preliminary stuff, I can talk about what I actually wanted to: git rebase (and the associated workflow).

Initially, git rebase seemed like a bad thing to me.  It allows you to rewrite the history in a repository, frequently to squish commits together.  To someone raised on more traditional version control, that seems like a terrible idea.  After all, history is sacred.  If I didn’t care about history, why would I bother using version control?

It is only recently that I’ve come to understand the cases where this is okay (and actually very beneficial).  I still think that one should never mess with the main history (and if there is a change that someone else has used the branch you are changing, then it is a terrible idea to change history).  But what git rebase does it let one work on a local branch and make as many little commits as possible.  Then, when the feature or bugfix is done, one can rebase against the main branch and make all that work into a single commit.  In source code with multiple commiters working on different features, this is really the only way to end up with history that makes any sense.  For a good counter-eample, one need only look at the git history for android.

That’s really all I wanted to point out, that when used with branches, git rebase can be a good way to maintain a legible history on the main branch for code (but I still don’t think it should ever be used on the main branch).  This also lets you avoid all sorts of ugly merge commits that don’t really say anything.

I’ve made some changes on this blog and hope to make some more soon (and maybe even post occasionally).

I’ve decided to switch from Blogger to WordPress hosted on my webserver.  I generally prefer to host myself whenever I can and Beth is planning on switching her blog to WordPress so it seemed like a good time to do it too

I’ll probably change change the theme around a bunch, but I kind of like WordPress so far (except that it is a bunch of nasty PHP).

I want to talk about compilers a bit.  I haven’t really worried about compilers in a few years, but since it sounds like there may be some interesting things going on in that area, it seemed like there might be something to look into again.

There’s a lot of talk these days about clang and llvm as a replacement for gcc. This has gained some momentum since Apple’s switching over to llvm (which ended up causing trouble for people who use MacPorts and lots of UNIX tools on the Mac, but that’s another story).  The various BSDs seem to have shown some interest in this too.

Ignoring the non-technical issues that seem to drive some people in this direction (I don’t really understand the anti-GNU sentiment that seems to come up relatively frequently these days either), llvm is supposed to be better at optimizing output than gcc.  This is where I actually get interested.

Most of my work on optimizing code comes from doing simulations in FORTRAN 77 when I was in grad school.  So the first thing I looked for to test this was a FORTRAN frontend for llvm.  This may seem like an odd thing to want, but lots of technical and scientific work is still done in FORTRAN.  It is actually a great language for a certain set of problems.  Turns out clang is pretty much for C-like things (I guess the name should have been a hint there).

To support languages like FORTRAN, there is a project from llvm called Dragon Egg.  It isn’t really a compiler, it is just a plugin to allow you to use llvm to do optimization for gcc (and since they claim FORTRAN support, presumable gfortran).

So, I installed it, checked out a copy of ZEUS-MP from my svn repo, modified the makefile, and tried to compile it.  And nothing.  I got strange build errors right away.  Now Zeus is extremely portable at this point.  In the past it was known to run on UNICOS, Irix, and lots of other long gone systems.  I’ve personally made sure that it could compile with the Intel Compilers, Portland Group Compilers, G77, GFortran, and Sun Studio Fortran Compiler.  I actually found the best performance with the gcc based compilers (g77 and gfortran).  Seemed surprising considering that in toy examples, the Intel Compilers generally perform best.  I have a figure or two related to this in my dissertation.

I could probably fight and try to get zeus to compile this way, but I don’t do hydrodynamics or FORTRAN for a living any more, and this was just supposed to be a quick experiment, so I don’t think I’ll put the time into it.  Maybe they will eventually get FORTRAN support to work better and I’ll try this again.  But for now, I’ll stick with the GNU Compilers.

I already posted on this topic specifically related to Angry Birds, but I figured it’s worth putting a more general version of it up.

I’ve seen a few reasons why one might need to restore the saved states of a game on an Android device.  This could be necessary when moving to a new device, updating the OS on a device, or restoring after some sort of failure (I’ve seen Angry Birds data get corrupted as well as Temple Run).

In principle, Google provides a way for developers to store information with the user’s Google account so data gets restores just like calendar and contact data. This would be nice, but it depends on the developer actually updating their device to use this relatively new Android feature (and I have yet to encounter a non-google app that uses it).

The startup FTW is working on a similar solution only it is platform independent as well as device independent. Even better, unlike the Google solution, this one actually lets you see what data you have (along with other cool features). (And to be totally transparent, I’ve had some involvement with the FTW guys, so I’m not 100% objective here).  This is a really nice way to deal with game data, but it still has the same issue where if a developer doesn’t use it, you data is trapped on a device.

So, until more developers use one of those two solutions (hopefully FTW) for backing up game states on mobile devices, Android uses need to take matters into their own hands.

I’ve tested this on a Droid Incredible, Droid, Xoom, and Galaxy Nexus.  I’ve done it with Android 2.3 and 4.0.  There is no need for the device to be rooted although that make some things easier.

Apps on Android can save files to their own directory on the /data partition and possibly to a similar directory on the sdcard if present (/mnt/sdcard/).

The first thing you need is the Android SDK (you really just need the adb tool that comes with it).  It can be downloaded (along with instructions on how to make it work) from the Android development site.

Once you have that installed on your computer and can see your device when plugged in with the command:

john@yoshi ~ $ adb devices

(Note, all of this must be done from a command line of some kind.  My examples were done in an xterm on GNU/Linux, but any terminal, including the DOS command prompt on Windows should work with at most minor changes).

It is now necessary to put your device “USB Debug Mode”.  The option for that is either in the Applications part of the settings or the Developer Options.  This varies with different versions of Android.

You now need to figure out the directory where the game is storing the data.  This name will match the java package name of the application (which isn’t actually made readily available anywhere).

Open a shell on the device:

adb shell

This will connect you directly to the device. Then type the command:

ls /data/data/

This will show a list of directories. You need to figure out the directory for the app you are looking for. Usually it will be obvious, although if not, google can help.

The apps I know for sure are:

  • Angry Birds: com.rovio.angrybirds
  • Angry Birds Rio: com.rovio.angrybirdsrio
  • Angry Birds Space: com.rovio.angrybirdsseasons
  • Angry Birds Space (Free): com.rovio.angrybirdsspace.ads
  • Cut The Rope: com.zeptolab.ctr.paid
  • Temple Run: com.imangi.templerun

Now check the sdcard in the same way:

ls /mnt/sdcard/Android/data

for a similar directory. Of the games I listed above, only Temple Run stores data on the sdcard.

Once you have come up with a list of files, for each one you must execute the command:

adb pull /data/data/PATHTOFILE LOCAL_PATH_TO_FILE

If you need to restore, the command is:
adb push LOCAL_PATH_TO_FILE /data/data/PATHTOFILE.

Generally, you need to start up the game (and in the case of Angry Birds) play a level so it will create its directories and save files before you put you backup copy back.

The files I know about are:

Angry Birds

/data/data/com.rovio.angrybirds/settings.lua
/data/data/com.rovio.angrybirds/highscores.lua

(Only the directory changes for the different angry birds versions.)

Cut the Rope

 /data/data/com.zeptolab.ctr.paid/databases/achievements.db
 /data/data/com.zeptolab.ctr.paid/databases/webviewCache.db
 /data/data/com.zeptolab.ctr.paid/databases/webview.db
 /data/data/com.zeptolab.ctr.paid/databases/scores.db
 /data/data/com.zeptolab.ctr.paid/databases/webviewCache.db-journal
 /data/data/com.zeptolab.ctr.paid/shared_prefs/com.scoreloop.achievements.store_bd7eceaa-009d-4d58-beb8-1753e5412b54_data.xml
 /data/data/com.zeptolab.ctr.paid/shared_prefs/com.scoreloop.achievements.store_bd7eceaa-009d-4d58-beb8-1753e5412b54.xml
 /data/data/com.zeptolab.ctr.paid/shared_prefs/com.scoreloop.ui.login.xml
 /data/data/com.zeptolab.ctr.paid/shared_prefs/CtrAppPaid.xml

Temple Run

/data/data/com.imangi.templerun/shared_prefs/P31Prefs.xml
/data/data/com.imangi.templerun/shared_prefs/RTA.xml
 /data/data/com.imangi.templerun/shared_prefs/com.imangi.templerun.xml
 /mnt/sdcard/Android/data/com.imangi.templerun/files/recordmanager.dat
 /mnt/sdcard/Android/data/com.imangi.templerun/files/SpaceHolder.dat

For other games, you’ll have to figure out the files yourself, but the general idea will always be the same.

All of the above can just as easily be done for apps other than games, I just haven’t found the need for it myself.

I would love to write a program to automate this process on the Android device itself (and either save the files to the sdcard or mail them to the user).  Unfortunately, the security policy on Android prevents this.  Each application is assigned a user name and group when installed.  It is the only application (with the possible exception of some system processes I think although I haven’t experimented with that enough) that has permission to read or write to its own directories and files.  Another app by the same developer (with the same signing key) can get permission to them, but that really doesn’t help me.  So on devices that haven’t been rooted, it seems that this method where the computer is also used is necessary.

I’ve been a member of the W3C html working group for a couple of years now. I don’t really participate as much as I would like, but I still think the group’s work is important.

So, with that in mind, I figured it was time to finally convert my website to html5. While technically it is still just a draft proposal, the fairly basic features are set, so a simple site like mine should be easy to convert.  I figured it might be worth putting what I did up here, even if just so I can see it later.

Up until now, I’ve always used XHTML. This means I kept presentational elements to a minimum and didn’t have the mess of tag soup that many older html documents contain.

The first thing was to change the mime-type the page is served as. xhtml has to be served with the application/xml+xhtml type to be valid (even though the broken IE browser can’t handle that). The correct type for html5 is simple text/html This can be done in your .htaccess file with the line:

RewriteRule \.html$ – [T=text/html]

Then, I replaced the top of each file (up to the title tag) with:

< !doctype html>

Other than that, I only had a few minor issues to fix.  I used the tag in one place (replaced it with css font-size:300%;), put the align properties for the tables in css

table     {
    margin-left:auto;
    margin-right:auto;
    text-align: center;
    width:100%;
    }

And finally since the name element is deprecated, any time I used it, I just replaced it with an id in a nearby tag (usually a heading tag, but sometimes a paragraph).

I haven’t yet dealt with this blog (but due to blogger’s messy templates, I never even got it to be valid xhtml, so this may not be easy). I also have a bunch of pages that are generated by a perl script. I modified the perl script, but just don’t feel like testing it tonight. But otherwise, goodbye XHTML, it was fun.  All told, the html5 pages are definitely smaller since there is less junk on the top of each page and I can’t be lazy and put styling all over the place, it has to go in the css.

(Note: I realize this is very simple, but I wasn’t able to find it by searching and therefore had to figure it out myself. Hopefully by posting it somewhere, I can save someone else the time. And if not someone else, maybe I’ll save myself the trouble next time it comes up.)
Recently, after an update for Angry Birds Seasons on Beth’s Motorola Droid, the app refused to open. I know that if I were to just reinstall it, all progress would be lost. I looked around online and I see other people have had similar problems with the various Angry Birds versions on android phones. I filed a bug report with Rovio but never heard back.
A little while later, the SD card on my phone had some issues. This causes both Angry Birds and Angry Birds Rio to stop working on my Droid Incredible. I could just reinstall them, but that would also cause me to lose all my progress. Turns out it wasn’t too hard to solve both problems.
Angry Birds stores its save data and settings in files on the phone in:

/data/data/com.rovio.angrybirds
/data/data/com.rovio.angrybirdsseasons
/data/data/com.rovio.angrybirdsrio

If your phone is rooted, then you can just access these files directory from a terminal on the phone. Unfortunately, if your phone isn’t rooted, you don’t have access to these files (and rooting some phones requires wiping them which would defeat the whole purpose of this).
There is a way around this. If you download the Android SDK from google, you can access these files without root permission on the phone. Install the SDK following the instructions from google. When you connect your phone, make sure to turn USB debugging mode on (it can be found in Settings->Applications->Development).
Once you are connected, you can use the adb tool to copy the Angry Birds files off the phone for safe keeping. You should get both the file settings.lua and highscores.lua. Assuming you are doing this with bash on GNU/Linux (or a Mac)

john@yoshi ~ $ cd tmp/
john@yoshi tmp $ GAME=angrybirds
john@yoshi tmp $ mkdir $GAME
john@yoshi tmp $ adb pull /data/data/com.rovio.$GAME/files/settings.lua ~/tmp/$GAME/
john@yoshi tmp $ adb pull /data/data/com.rovio.$GAME/files/highscores.lua ~/tmp/$GAME/

(And the repeat with GAME=angrybirdsrio and GAME=angrybirdsseasons.)
After doing that on my phone, I reinstalled all three Angry Birds on my phone and then copied the files back with the command:

john@yoshi ~ $ GAME=angrybirds
john@yoshi ~ $ adb push ~/tmp/$GAME/settings.lua /data/data/com.rovio.$GAME/files/
john@yoshi ~ $ adb push ~/tmp/$GAME/highscores.lua /data/data/com.rovio.$GAME/files/

(Again, repeating for each game.)
For Beth’s phone, the pull command could not find the settings file for Seasons. It seems that the settings file was erased during the update. Just putting a blank file back was enough to fix it! All the settings were lost, but there aren’t many settings so that didn’t really matter since the level data was saved.
So, now I back up my Angry Birds data every once in a while. I haven’t had a chance to really examine the save files (.lua) but since I have copies, I’ll probably take a look at some point.

I found this while going through some computer parts.  Makes me think that perhaps I keep old computer stuff just a little too long.

Just trying out the Android app for Blogger.  It actually seems pretty nice.  Maybe it will convince me to actually use this blog again.
The only complaint I have so far is that it doesn’t let me pick labels from a list which is a problem considering the fact that I don’t remember any of the labels I’ve used in the past.

I haven’t posted any code in way too long.

I hope to write more about org-mode real soon since I am very impressed with it (and it is nice to take care of one of my more important computing tasks (todo lists) in emacs) but for now, here is the blurb about this python code from my download page. I realize I could probably do this using emacs lisp, but I’m a bit (or more than a bit) better with python than with lisp.

I depend on my todo lists pretty heavily. At one point I used the
palm todo list (and jpilot for syncing on my computer). Eventually
I switched to Apple’s iCal (I could never get Mozilla Sunbird to
work reliably enough). This was kind of annoying since I’m not
really a fully time Mac user (I still prefer GNU/Linux or at least
UNIX cmd line stuff). For some things (grad school and work) I used
some perl/LaTeX thing which is a bit cumbersome. So I was pretty
excited by Emacs org-mode. I wrote some python to convert my ical
files to emacs org-mode. I tried to make it general enough that the
todo class would be a good start for a python-ical interface. If I
feel motivated, it should be fairly easy to expand this into something
more. For now, it is just to get iCal todo items into a python data
structure and can write it to orgmode todo items.

read_ical.py

I’m not exactly what you would call a “good” blogger. I can go months without actually posting anything (I blame facebook and twitter for taking care of my desire to talk to myself). So I found it a little odd when I started seeing a few comments on some old posts.

I didn’t delete them immediately because I didn’t initially understand them. In one case, there was a comment from an Anonymous user on a two year old posting about video game consoles. The funny thing was the comment was sort of on topic and had no links in it. Then a day latter there was another similar comment.

Finally, a day after that there was a post that starts out by apologizing for being off topic and has some sketchy looking links in it. The links have their href tags (or some attempt at a blogger shortcut for that) messed up which does not help make it any less sketchy.

Then next day, another identical copy of the same comment shows up (and I hadn’t removed the first one yet at this point).

I’ve removed those comments, but it seems like a lot of work just to try and slip in a few sketchy links onto a blog that gets basically zero traffic.

Just makes me wonder if a tiny boost in page rank is really worth that much to people.

Of course it would be really funny if I was wrong and those were actual people, in which case I would have some apologizing to do, but I doubt it.