The Daver Was Geek

For my work, I use Zimbra Desktop as my email app; it's a pretty decent UI and provides all the features I need for a 'business' email tool.

BUT. It uses a bundled, ultra-defunct Prism install as the UI container. And now that Oneiric Ocelot has dropped GTK2, that lovely container just won't run.

Since I primarily use Chrome as my browser these days, I would be fine with having Zimbra run inside Chrome, but having it as an "App Tab" is still not ideal. I want to keep my email running in a chromeless window on a separate monitor, which having it in the Prism container would do perfectly.

Enter Chrome's App Mode -- call google-chrome with --app=$url, where url is the url of the webapp you want to use, and it will happily give you a fresh window with no chrome -- full-window App goodness!

Now, Zimbra Desktop's architecture includes two pieces: a Jetty-based server (which acts as a local synchronization store and serves the Web UI), and the Prism UI container. The Jetty server, upon startup, writes a URI into a webapp.ini file, which Prism uses as the startup URL.

So. I need to make sure the Jetty server is running, or there won't be a URI to go to. Luckily it writes a pidfile when it starts up, and removes it when it shuts down, so a naive approach is to just check for that file's existence.

Then, I need to grab the URI from the webapp.ini file (it's unique to a given instance of the Jetty server), pass it to Chrome, and bam! I'm in a new full-frame Zimbra Desktop window!

Here's my script:

#! /bin/sh

USERROOT=$HOME/.zdesktop

if [ ! -f $USERROOT/log/zdesktop.pid ];
then
    echo "Starting Service..."
    $USERROOT/bin/zdesktop start;
    sleep 5;
fi

url=`grep uri $USERROOT/zdesktop.webapp/webapp.ini | cut -d= -f2-`
echo $url

/opt/google/chrome/google-chrome "--app=$url"

UPDATE: This code is now hosted on Bitbucket.

And there you have it. Not sure if anyone else cares, but if you do...enjoy!

As I've been doing more code for Band Back Together, I decided to post my hacks and snippets on BitBucket. This is giving me a good opportunity to spend some more time with Mercurial, which is a Python-based distributed VCS.

I've been following the various VCS tools that have come of age as part of the open-source movement, and while I've used them for the purpose of pulling down code and hacking on it, I've only ever sent patches to maintainers and used Subversion for my own code, never really relying on the 'new' approach of DVCS. Now I see what all the fuss is about.

With Mercurial, I can work FAST. Really fast. It handles mass changes -- say, a bulk sed change to a constant name, or a refactoring effort -- with ease; I just do my work and when a change seems complete I commit it. It takes just a moment, and there I am -- on to the next change. So the version control is just a natural finger-memory activity; I spend effectively zero time on it incrementally. For web development, I init a repository in my development webroot -- then I can merge in changes or edit/test on my local dev site directly. Then, for integration, I simply pull the changes into the integration site. Until I'm ready to cut a release, I again spend next to no time doing version control tasks, but I still keep my entire revision history ( using my commit-early-commit-often mantra).

Validating a change on multiple branches? Trivial. Switching between branches is fast enough that I don't think twice about it.

So what's the downside here?

Well, there's always a tradeoff. Sure, the push/pull mechanism is simple and fast -- but now I need some sort of server set up (and yes, hg includes one, but if I was in an organization that did not allow such freedoms, it limits the usefulness pretty quickly). Of course, you can set up a central server which accepts pushes from entire groups, but then you *definitely* need gatekeepers who review and merge regularly. And because each repository is standalone, the other edge of the sword is that you have to set up your own controls around the 'master' repository; in order to ensure your testing is not compromised by sloppy commits, you need to set up processes that handle the complexities for your workflow. Which also means that your master repo owners need to be well-trained in how the VCS works.

Contrast this with ClearCase, which I use professionally, and have built a love-hate relationship with. ClearCase needs a lot of hardware to be fast. It does not particularly care for distributed development and requires connectivity to a server to do any activity. It also treats each file or directory as a separate object -- which means you must do a checkout on each. file. you. want. to. change. and then check it back in. The overhead of all this leads devs to avoid committing until post-testing, so they can avoid the expense of the check-in/out cycle. (most devs are lazy and impatient -- so this makes total sense)

Management, however, loves ClearCase. They can see a central repository, backed-up, controlled, which has Their Intellectual Property carefully maintained. It has a 'dynamic view' feature which essentially provides a live, revision-controlled filesystem. It has decent-looking Windows/Visual Studio plugins, and hooks up to ClearQuest for issue tracking. They can define triggers which ensure coding standards or additional security or repository structure or basically anything -- so providing a technical solution to controlling the quality of the repository, and allowing non-coders to manage a repository.

In practice, I would love to integrate the two systems -- using Mercurial for day-to-day fast coding, then pulling the completed work into ClearCase on a per-issue basis -- i.e. the final, tested result, which corresponds to a ClearQuest ticket, is what is committed in ClearCase -- so management sees the issues resolved with code commits, and the developers can collaborate, hack, branch, merge, and refactor at high speed day-to-day without the overhead.

Meantime, for open-source and distributed projects which don't need that formal control scheme -- I am thrilled with Mercurial. If you haven't seen it, definitely give it a spin.

I do the tech stuff for my wife’s various websites, which mainly run on WordPress. So even though I am kind of against PHP as a matter of principle, I have had to learn how to bend WordPress to my will in a number of ways.

Lately, there have been a number of requests to change or add features to a live site — which means I immediately must go and set up a dev site so that I can hack away at it without fear of breakage.

Setting up the basic infrastructure is no biggie — apt-get the necessary apache2 and php packages, set up mysql, etc.

Now comes the fun part: getting the dev site to behave just like the live one. To do this, we need two things:

• all the PHP code that’s been modified/hacked
• A complete copy of the MySQL database (slightly modified)

So let’s take care of the first step. For this, we can use the file-transfer Swiss Army knife: rsync.

rsync -e ssh -u -r --filter="- cache/***" -i user@web.host.net:/path/to/site /pathto/devsite-root/

Breaking it down:

The -e ssh means we want to use ssh as our remote shell.

The -u tells rsync that we want to update from the prod site — but since we’re doing development, we don’t want to blow away our changes. So any file with a newer time stamp on the receiving side gets left alone.

The -r says to operate recursively.

The --filter="- cache/***" tells rsync to filter any directory named ‘cache’ and any files beneath a directory so named. Since the live site uses WP-SuperCache, this saves us from downloading the cache files.

Finally, the -i tells rsync to spit out what files it changed. After the first run, there shouldn’t be too much. :)

Now, if all went well, you have the latest and greatest copy of the wordpress installation on your DEV host. I leave it to you to set up Apache to point to it.

NEXT, we need to grab the database. Again, I leave it to you to set up MySQL. But one thing: use the same database name on the dev side as the prod side. It saves some headaches. Oh, and one other thing: you might need to have a .my.cnf and/or ssh authorized_keys set up for passwordless use; I trust you can figure out how to do that.

Thanks to ssh and mysqldump, we have a way to pipe the data over a secure channel directly into our dev database. I wouldn’t recommend this approach for really large databases, but for our relatively small Wordpress sites it’s good enough:

ssh bkup@web.host.net 'mysqldump dbname | gzip' | gzip -d | mysql dbname

Easy! We enclose the remote command 'mysqldump dbname | gzip' in quotes, then pipe the result of that remote command to a local gzip -d and pipe the decompressed dump directly into the local database!

One little problem, though: now the DEV database has pointers to the PROD URL — and if you test anything, you end up touching your prod site.

So I wrote a little SQL which switches things over to DEV settings:

update wp_options set option_value = 'http://dev.mysite.com' where option_name='siteurl';
update wp_options set option_value = 'http://dev.mysite.com' where option_name='home';
update wp_options set option_value = 'Dev Site Name' where option_name='blogname';

And we’re done! Ready to hack and slash.

I’ve had Clear for a while now, and have been using it on my commute with my netbook, an HP Mini. The connection is good with the exception of a few dead spots; however there’s one big catch for me: it only works on Windows.

So ever since I got it, I’ve been casting about for an opportunity to get things to work with Linux. I know I could ditch my Ubee modem for a Clear Spot, but then I have yet ANOTHER battery-powered device to deal with, not to mention that I’ve already paid for this modem. Besides, what kind of a geek would I be if I wasn’t trying to figure stuff out?

I’ve been following this thread for some time, and finally someone picked up on a location of some of the open source code for Beceem chipsets — which are commonly being used for the 4g USB modems.

I grabbed the package, which indeed does include a driver and connection manager for Beceem chipsets, along with raft of documentation.

I built everything according to the docs, ended up with a kernel module (drxvi314.ko), an SSL plug-in library ( libengine_beceem.so), some various tools and udev rules, and a client-server (wimaxc/wimaxd) connection manager.

I inserted the module, plugged in my Ubee PXU1900 and….

Nothing. Of course it wouldn’t be that easy.

So I started from some lsusb output:

Bus 001 Device 003: ID 0489:e016 Foxconn / Hon Hai

I started taking a look at the driver source — since it’s a USB device, there had to be some sort of reference to the vendor and product IDs somewhere, so the driver knows which devices it knows how to work with.

And sure, enough, in InterfaceInit.c:

static struct usb_device_id InterfaceUsbtable[] = {
    { USB_DEVICE(BCM_USB_VENDOR_ID_T3, BCM_USB_PRODUCT_ID_T3) },
    { USB_DEVICE(BCM_USB_VENDOR_ID_T3, BCM_USB_PRODUCT_ID_T3B) },
    { USB_DEVICE(BCM_USB_VENDOR_ID_T3, BCM_USB_PRODUCT_ID_T3L) },
    {}
};

And then I noticed that there was already a patch in the package to add support for some more devices:

--- ./Interface/usb/InterfaceInit.c.dist    2010-08-19 13:10:50.000000000 -0500
+++ ./Interface/usb/InterfaceInit.c 2010-08-19 13:23:30.000000000 -0500
@@ -4,6 +4,8 @@
     { USB_DEVICE(BCM_USB_VENDOR_ID_T3, BCM_USB_PRODUCT_ID_T3) },
    { USB_DEVICE(BCM_USB_VENDOR_ID_T3, BCM_USB_PRODUCT_ID_T3B) },
    { USB_DEVICE(BCM_USB_VENDOR_ID_T3, BCM_USB_PRODUCT_ID_T3L) },
+   { USB_DEVICE(BCM_USB_VENDOR_ID_ZTE, BCM_USB_PRODUCT_ID_226) },
+   { USB_DEVICE(BCM_USB_VENDOR_ID_FOXCONN, BCM_USB_PRODUCT_ID_1901) },
     {}
 };

So I looked up where BCM_USB_PRODUCT_ID_1901 was coming from, and found it in InterfaceInit.h:

#define BCM_USB_PRODUCT_ID_1901 0xe017

Hey, what do you know? It’s only one off from the e016 value reported by lsusb! I added one more constant to the header:

#define BCM_USB_PRODUCT_ID_1900 0xe016

And listed that vendor/product pair in the InterfaceInit.c file:

{ USB_DEVICE(BCM_USB_VENDOR_ID_FOXCONN, BCM_USB_PRODUCT_ID_1900) },

I compiled, inserted the module, and…BLINKING LIGHTS! Little Ubee lit up green!

I took a look at /var/log/messages and…

Sep 24 21:20:52 hokusai kernel: [ 3447.228101] usb 1-2: reset high speed USB device using ehci_hcd and address 3
    Sep 24 21:20:53 hokusai kernel: [ 3448.201279] Do Post chip reset setting here if it is requiredddr_init:Register Count is =48
    Sep 24 21:20:53 hokusai kernel: [ 3448.207521]
    Sep 24 21:20:53 hokusai kernel: [ 3448.221020] InitCardAndDownloadFirmware:CFG file downloaded
    Sep 24 21:20:53 hokusai kernel: [ 3448.221763]
    Sep 24 21:20:53 hokusai kernel: [ 3448.221768] /usr/src/Rel_5.2.7.3P1_USB/Source/Driver/Network/OSAL/Linux/usb/Common/nvm.c:ReadMacAddressFromNVM:420:Buffer dump of size 0x6 in the HEX:
    Sep 24 21:20:53 hokusai kernel: [ 3448.221782] 00 1D 88 09 8A 42
    Sep 24 21:20:53 hokusai kernel: [ 3448.221804] register_networkdev:Registering netdevice notifier
    Sep 24 21:20:53 hokusai kernel: [ 3448.221812]
    Sep 24 21:20:53 hokusai kernel: [ 3448.221826] register_networkdev:BCM Notifier got Registered
    Sep 24 21:20:53 hokusai kernel: [ 3448.221859]  ========= bcm_init (noop) ======== bcm_notify_event:Register RefCount: 5
    Sep 24 21:20:53 hokusai kernel: [ 3448.225736]
    Sep 24 21:20:53 hokusai kernel: [ 3448.226497] register_networkdev:Beceem Network device name is eth2!
    Sep 24 21:20:54 hokusai kernel: [ 3448.506768] open_firmware_file:Got file descriptor pointer of /lib/firmware/macxvi200.bin!
    Sep 24 21:20:54 hokusai kernel: [ 3448.506792] BcmFileDownload:Opened file is = /lib/firmware/macxvi200.bin and length =0x1cd120 to be downloaded at =0xbfc00000
    Sep 24 21:20:54 hokusai kernel: [ 3448.506810] BcmFileDownload:download start 46b16b79
    Sep 24 21:20:54 hokusai kernel: [ 3448.753786] InterfaceFileDownload:Got end of file!
    Sep 24 21:20:54 hokusai kernel: [ 3448.993528] InterfaceFileReadbackFromChip:Got end of file!
    Sep 24 21:20:54 hokusai kernel: [ 3448.993554] BcmFileDownload:file download done at 46b16d60
    Sep 24 21:20:54 hokusai kernel: [ 3448.993561] InitCardAndDownloadFirmware:BIN file downloaded
    Sep 24 21:20:54 hokusai kernel: [ 3449.003695] device_run:Sending first interrupt URB down......
    Sep 24 21:20:54 hokusai kernel: [ 3449.006946] device_run:Got the mailbox interrupt ...Registering control interface...
    Sep 24 21:20:54 hokusai kernel: [ 3449.006955]
    Sep 24 21:20:54 hokusai kernel: [ 3449.007333] register_control_device_interface:Got Major No: 250
    Sep 24 21:20:54 hokusai kernel: [ 3449.012484] usbcore: registered new interface driver usbbcm

It’s aliiiiiive! It even grabbed its firmware! We are oh so close.

Next it was time to get the connection manager working. Here’s where my limited knowledge of WiMAX is starting to hurt.

The first part was simple enough: build the connection manager, and start up the server component (details on this are in the docs inside the package):

$wimaxd -c /etc/Sprint4g/wimaxd.conf

Then fire up a client and hook it up:

$wimaxc -i
Beceem CM Server Version 1.1.6.0

>

I tried a ‘search’ (one of the commands in the ‘help’:

Network search returned 2 base stations.
Idx BSID                    Pre      Freq      BW  RSSI  CINR
0  01:01:00:00:02:26:42:f2 0x37 2647.000  10.000   -73    24
1  01:01:00:00:02:26:42:f3 0x57 2667.000  10.000   -86     7

Now that looks good — two different base stations to play with. At this point, I’m thinking I’m in really good shape! So let’s try to connect!

> connect
Connect request submitted for:
    Center frequency    2647.000 MHz
    Bandwidth           10.000 MHz
    Preamble index      0x37

fingers totally crossed sixteen ways

…and nothing.

Well, not nothing, exactly, but not something I know what to do with:

--------------------- EAP Authentication ------------------
Authentication failed (key generation)
 -----------------------------------------------------------

From what I can glean of how this is working, the network is trying to use EAP-TLS to authenticate my device on the network. To do that, it needs my device certificate; however I see no such cert in the distribution package on Windows which leads me to guess that it’s on the non-volatile memory of the device itself. Encrypted, of course.

The key is likely also stored on the device. Convenient. BUT. It needs a passphrase.

I’m trying the approaches suggested in the Sprint docs from the package, including generating the passphrase based on Mac ID; but I am stuck on the above — I can’t seem to get a valid key.

So I post this in the hope that someone who knows more about EAP and device certificates might be willing/able to lend a Daver a hand. My lit-up Ubee is staring at me, wanting so badly to be connected…