Skip navigation

Category Archives: computers

Not much to say here, just a test posting after moving this site to a new webserver. I may try to write up some of the details needed to get WordPress and OpenBSD to play nicely together if I have time later.

I’ve been using git for version control for a few years now and at several different companies (although I still use subversion for personal projects).  I think I finally get it and understand the common workflow with it so I wanted to write it down.  This largely involves the rebase command (a command which I was initially weary of), hence the post title.

For starters, unlike many people these days, I don’t think git is absolutely superior to other version control systems.  As I said, I still use subversion for my personal stuff.  So let’s start with some negatives.

First, git doesn’t store directories or history about them.  So if you you want an empty directory, you need to add a hidden file (.gitignore is a good choice) to the “empty” directory.  As someone who moved from cvs to subversion, this feels like a step backwards.  At least in cvs, I understand that it is an unavoidable artifact of using rcs files, but with git I really don’t understand the decision.

Second, git is built very much around the idea of a single project per repository.  There are situations where this makes a lot of sense, but there are use cases where I would prefer to have multiple things in one repository and just check out the parts you want.

Finally, I miss cvs tags in files (which subversion supports just fine).

I’m in a good mood so I won’t count the documentation against git.

One of the things people always say is that git is great because it is decentralized and I’m not sure I buy that.  In every place I’ve worked (and every major project using it that I’ve looked at), there is always one master, central, blessed repository that everyone else clones and pushes to (sometimes indirectly if only limited people have access).  I think this makes sense, but it makes the decentralized talk seem a little funny.

The other part of it being decentralized is that you have a complete backup of the repo with every clone.  Except that git clone only clones some things.  You can fix this (more or less) with:
$ git fetch --all --tags

Now that I’m done with all that preliminary stuff, I can talk about what I actually wanted to: git rebase (and the associated workflow).

Initially, git rebase seemed like a bad thing to me.  It allows you to rewrite the history in a repository, frequently to squish commits together.  To someone raised on more traditional version control, that seems like a terrible idea.  After all, history is sacred.  If I didn’t care about history, why would I bother using version control?

It is only recently that I’ve come to understand the cases where this is okay (and actually very beneficial).  I still think that one should never mess with the main history (and if there is a change that someone else has used the branch you are changing, then it is a terrible idea to change history).  But what git rebase does it let one work on a local branch and make as many little commits as possible.  Then, when the feature or bugfix is done, one can rebase against the main branch and make all that work into a single commit.  In source code with multiple commiters working on different features, this is really the only way to end up with history that makes any sense.  For a good counter-eample, one need only look at the git history for android.

That’s really all I wanted to point out, that when used with branches, git rebase can be a good way to maintain a legible history on the main branch for code (but I still don’t think it should ever be used on the main branch).  This also lets you avoid all sorts of ugly merge commits that don’t really say anything.

I want to talk about compilers a bit.  I haven’t really worried about compilers in a few years, but since it sounds like there may be some interesting things going on in that area, it seemed like there might be something to look into again.

There’s a lot of talk these days about clang and llvm as a replacement for gcc. This has gained some momentum since Apple’s switching over to llvm (which ended up causing trouble for people who use MacPorts and lots of UNIX tools on the Mac, but that’s another story).  The various BSDs seem to have shown some interest in this too.

Ignoring the non-technical issues that seem to drive some people in this direction (I don’t really understand the anti-GNU sentiment that seems to come up relatively frequently these days either), llvm is supposed to be better at optimizing output than gcc.  This is where I actually get interested.

Most of my work on optimizing code comes from doing simulations in FORTRAN 77 when I was in grad school.  So the first thing I looked for to test this was a FORTRAN frontend for llvm.  This may seem like an odd thing to want, but lots of technical and scientific work is still done in FORTRAN.  It is actually a great language for a certain set of problems.  Turns out clang is pretty much for C-like things (I guess the name should have been a hint there).

To support languages like FORTRAN, there is a project from llvm called Dragon Egg.  It isn’t really a compiler, it is just a plugin to allow you to use llvm to do optimization for gcc (and since they claim FORTRAN support, presumable gfortran).

So, I installed it, checked out a copy of ZEUS-MP from my svn repo, modified the makefile, and tried to compile it.  And nothing.  I got strange build errors right away.  Now Zeus is extremely portable at this point.  In the past it was known to run on UNICOS, Irix, and lots of other long gone systems.  I’ve personally made sure that it could compile with the Intel Compilers, Portland Group Compilers, G77, GFortran, and Sun Studio Fortran Compiler.  I actually found the best performance with the gcc based compilers (g77 and gfortran).  Seemed surprising considering that in toy examples, the Intel Compilers generally perform best.  I have a figure or two related to this in my dissertation.

I could probably fight and try to get zeus to compile this way, but I don’t do hydrodynamics or FORTRAN for a living any more, and this was just supposed to be a quick experiment, so I don’t think I’ll put the time into it.  Maybe they will eventually get FORTRAN support to work better and I’ll try this again.  But for now, I’ll stick with the GNU Compilers.

I already posted on this topic specifically related to Angry Birds, but I figured it’s worth putting a more general version of it up.

I’ve seen a few reasons why one might need to restore the saved states of a game on an Android device.  This could be necessary when moving to a new device, updating the OS on a device, or restoring after some sort of failure (I’ve seen Angry Birds data get corrupted as well as Temple Run).

In principle, Google provides a way for developers to store information with the user’s Google account so data gets restores just like calendar and contact data. This would be nice, but it depends on the developer actually updating their device to use this relatively new Android feature (and I have yet to encounter a non-google app that uses it).

The startup FTW is working on a similar solution only it is platform independent as well as device independent. Even better, unlike the Google solution, this one actually lets you see what data you have (along with other cool features). (And to be totally transparent, I’ve had some involvement with the FTW guys, so I’m not 100% objective here).  This is a really nice way to deal with game data, but it still has the same issue where if a developer doesn’t use it, you data is trapped on a device.

So, until more developers use one of those two solutions (hopefully FTW) for backing up game states on mobile devices, Android uses need to take matters into their own hands.

I’ve tested this on a Droid Incredible, Droid, Xoom, and Galaxy Nexus.  I’ve done it with Android 2.3 and 4.0.  There is no need for the device to be rooted although that make some things easier.

Apps on Android can save files to their own directory on the /data partition and possibly to a similar directory on the sdcard if present (/mnt/sdcard/).

The first thing you need is the Android SDK (you really just need the adb tool that comes with it).  It can be downloaded (along with instructions on how to make it work) from the Android development site.

Once you have that installed on your computer and can see your device when plugged in with the command:

john@yoshi ~ $ adb devices

(Note, all of this must be done from a command line of some kind.  My examples were done in an xterm on GNU/Linux, but any terminal, including the DOS command prompt on Windows should work with at most minor changes).

It is now necessary to put your device “USB Debug Mode”.  The option for that is either in the Applications part of the settings or the Developer Options.  This varies with different versions of Android.

You now need to figure out the directory where the game is storing the data.  This name will match the java package name of the application (which isn’t actually made readily available anywhere).

Open a shell on the device:

adb shell

This will connect you directly to the device. Then type the command:

ls /data/data/

This will show a list of directories. You need to figure out the directory for the app you are looking for. Usually it will be obvious, although if not, google can help.

The apps I know for sure are:

  • Angry Birds: com.rovio.angrybirds
  • Angry Birds Rio: com.rovio.angrybirdsrio
  • Angry Birds Space: com.rovio.angrybirdsseasons
  • Angry Birds Space (Free): com.rovio.angrybirdsspace.ads
  • Cut The Rope: com.zeptolab.ctr.paid
  • Temple Run: com.imangi.templerun

Now check the sdcard in the same way:

ls /mnt/sdcard/Android/data

for a similar directory. Of the games I listed above, only Temple Run stores data on the sdcard.

Once you have come up with a list of files, for each one you must execute the command:

adb pull /data/data/PATHTOFILE LOCAL_PATH_TO_FILE

If you need to restore, the command is:
adb push LOCAL_PATH_TO_FILE /data/data/PATHTOFILE.

Generally, you need to start up the game (and in the case of Angry Birds) play a level so it will create its directories and save files before you put you backup copy back.

The files I know about are:

Angry Birds

/data/data/com.rovio.angrybirds/settings.lua
/data/data/com.rovio.angrybirds/highscores.lua

(Only the directory changes for the different angry birds versions.)

Cut the Rope

 /data/data/com.zeptolab.ctr.paid/databases/achievements.db
 /data/data/com.zeptolab.ctr.paid/databases/webviewCache.db
 /data/data/com.zeptolab.ctr.paid/databases/webview.db
 /data/data/com.zeptolab.ctr.paid/databases/scores.db
 /data/data/com.zeptolab.ctr.paid/databases/webviewCache.db-journal
 /data/data/com.zeptolab.ctr.paid/shared_prefs/com.scoreloop.achievements.store_bd7eceaa-009d-4d58-beb8-1753e5412b54_data.xml
 /data/data/com.zeptolab.ctr.paid/shared_prefs/com.scoreloop.achievements.store_bd7eceaa-009d-4d58-beb8-1753e5412b54.xml
 /data/data/com.zeptolab.ctr.paid/shared_prefs/com.scoreloop.ui.login.xml
 /data/data/com.zeptolab.ctr.paid/shared_prefs/CtrAppPaid.xml

Temple Run

/data/data/com.imangi.templerun/shared_prefs/P31Prefs.xml
/data/data/com.imangi.templerun/shared_prefs/RTA.xml
 /data/data/com.imangi.templerun/shared_prefs/com.imangi.templerun.xml
 /mnt/sdcard/Android/data/com.imangi.templerun/files/recordmanager.dat
 /mnt/sdcard/Android/data/com.imangi.templerun/files/SpaceHolder.dat

For other games, you’ll have to figure out the files yourself, but the general idea will always be the same.

All of the above can just as easily be done for apps other than games, I just haven’t found the need for it myself.

I would love to write a program to automate this process on the Android device itself (and either save the files to the sdcard or mail them to the user).  Unfortunately, the security policy on Android prevents this.  Each application is assigned a user name and group when installed.  It is the only application (with the possible exception of some system processes I think although I haven’t experimented with that enough) that has permission to read or write to its own directories and files.  Another app by the same developer (with the same signing key) can get permission to them, but that really doesn’t help me.  So on devices that haven’t been rooted, it seems that this method where the computer is also used is necessary.

(Note: I realize this is very simple, but I wasn’t able to find it by searching and therefore had to figure it out myself. Hopefully by posting it somewhere, I can save someone else the time. And if not someone else, maybe I’ll save myself the trouble next time it comes up.)
Recently, after an update for Angry Birds Seasons on Beth’s Motorola Droid, the app refused to open. I know that if I were to just reinstall it, all progress would be lost. I looked around online and I see other people have had similar problems with the various Angry Birds versions on android phones. I filed a bug report with Rovio but never heard back.
A little while later, the SD card on my phone had some issues. This causes both Angry Birds and Angry Birds Rio to stop working on my Droid Incredible. I could just reinstall them, but that would also cause me to lose all my progress. Turns out it wasn’t too hard to solve both problems.
Angry Birds stores its save data and settings in files on the phone in:

/data/data/com.rovio.angrybirds
/data/data/com.rovio.angrybirdsseasons
/data/data/com.rovio.angrybirdsrio

If your phone is rooted, then you can just access these files directory from a terminal on the phone. Unfortunately, if your phone isn’t rooted, you don’t have access to these files (and rooting some phones requires wiping them which would defeat the whole purpose of this).
There is a way around this. If you download the Android SDK from google, you can access these files without root permission on the phone. Install the SDK following the instructions from google. When you connect your phone, make sure to turn USB debugging mode on (it can be found in Settings->Applications->Development).
Once you are connected, you can use the adb tool to copy the Angry Birds files off the phone for safe keeping. You should get both the file settings.lua and highscores.lua. Assuming you are doing this with bash on GNU/Linux (or a Mac)

john@yoshi ~ $ cd tmp/
john@yoshi tmp $ GAME=angrybirds
john@yoshi tmp $ mkdir $GAME
john@yoshi tmp $ adb pull /data/data/com.rovio.$GAME/files/settings.lua ~/tmp/$GAME/
john@yoshi tmp $ adb pull /data/data/com.rovio.$GAME/files/highscores.lua ~/tmp/$GAME/

(And the repeat with GAME=angrybirdsrio and GAME=angrybirdsseasons.)
After doing that on my phone, I reinstalled all three Angry Birds on my phone and then copied the files back with the command:

john@yoshi ~ $ GAME=angrybirds
john@yoshi ~ $ adb push ~/tmp/$GAME/settings.lua /data/data/com.rovio.$GAME/files/
john@yoshi ~ $ adb push ~/tmp/$GAME/highscores.lua /data/data/com.rovio.$GAME/files/

(Again, repeating for each game.)
For Beth’s phone, the pull command could not find the settings file for Seasons. It seems that the settings file was erased during the update. Just putting a blank file back was enough to fix it! All the settings were lost, but there aren’t many settings so that didn’t really matter since the level data was saved.
So, now I back up my Angry Birds data every once in a while. I haven’t had a chance to really examine the save files (.lua) but since I have copies, I’ll probably take a look at some point.

I found this while going through some computer parts.  Makes me think that perhaps I keep old computer stuff just a little too long.

I haven’t posted any code in way too long.

I hope to write more about org-mode real soon since I am very impressed with it (and it is nice to take care of one of my more important computing tasks (todo lists) in emacs) but for now, here is the blurb about this python code from my download page. I realize I could probably do this using emacs lisp, but I’m a bit (or more than a bit) better with python than with lisp.

I depend on my todo lists pretty heavily. At one point I used the
palm todo list (and jpilot for syncing on my computer). Eventually
I switched to Apple’s iCal (I could never get Mozilla Sunbird to
work reliably enough). This was kind of annoying since I’m not
really a fully time Mac user (I still prefer GNU/Linux or at least
UNIX cmd line stuff). For some things (grad school and work) I used
some perl/LaTeX thing which is a bit cumbersome. So I was pretty
excited by Emacs org-mode. I wrote some python to convert my ical
files to emacs org-mode. I tried to make it general enough that the
todo class would be a good start for a python-ical interface. If I
feel motivated, it should be fairly easy to expand this into something
more. For now, it is just to get iCal todo items into a python data
structure and can write it to orgmode todo items.

read_ical.py

I understand the desire to make error messages more friendly, but I’m not entirely sure about this one:

[ 297.708539] Uhhuh. NMI received for unknown reason 3d on CPU 0.
[ 297.708556] Do you have a strange power saving mode enabled?
[ 297.708558] Dazed and confused, but trying to continue

That would be the end of the output from dmesg on my Ubuntu desktop after I restarted it from a strange failure. That message was the only interesting thing in the log but sadly, it doesn’t really help.

Since I haven’t written anything in months I thought I would put something semi-informative here.

I’ve been slowly capturing all my family’s VHS home movies onto my computer and making (even slower then the capturing) DVDs. My main goal right now is just to get everything digital since the VHS aren’t exactly going to improve as they age (and some of them already look kind of bad). I may describe how I capture later but right now I want to talk about burning DVDs a little.

I make the DVDs on my Mac (a G4 powerbook running Leopard). I was trying to burn DVDs of three tapes before last weekend and ran into some very annoying trouble. I used iMovie and iDVD to make menus and all that and then made a disk image with iDVD. Then I tried to burn the dmg file using Disk Utility.app (the standard Apple thing).

The disk got through the burning fine but then would fail in the verify step every time. This lead to a pile of discarded DVDs before I looked into it closer. Disk Utility.app gave no useful info so I checked the log file it pointed it. And for each failed verify there was an error

0x80020063

Oddly, these failing DVDs all played just fine in a DVD player or a computer. Thanks to the error code this seemed like something I could google pretty easily. But I was wrong. Some people claimed it was due to bad media. Others claimed you need to repair disk permissions. A few said it was because of corrupted plist files (but they wouldn’t say which one). Made me really miss how useful the gentoo forums were where people actually know things and will share info.

In the end I tried the permissions and plist things with no luck. Finally, I tried burning a different brand of DVD and amazingly, it would verify fine. I’m not sure why the FujiFilm DVD-R fail at the verify step while the other disks worked. It could be related to the brand. Or it could be related to the -R and +R difference. I don’t have enough different disks to figure that out.

It seems to me that the verification error 0x80020063 is pretty much worthless since disks can fail that and still work fine. I’m not thrilled with this conclusion, but for now it’s all I’ve got.

After at least a month with no updates, I finally decided to do an emerge world to update ganon, my desktop. Generally I’ve found with gentoo that updates work great if you do them regularly, but wait to long and let too many build up and bad things can happen. A month or so definitely puts me in dangerous waters.

So when I saw the results of the emerge –sync that I have as cronjob, I decided nearly 200 packages to update was probably as far as I could push it before finally updating.

Before leaving this morning I started an update, and other than a minor problem with apache (due to a change in the package which they explained how to deal with very well), everything worked. New kernel and all. Considering all the talk gentoo’s problems have been getting, I was worried, but it seems to have been for nothing.