ryan kennedy’s blog

The topic of complex vs. complicated came up a lot at my last job (EarthLink). I was often subjected to long tirades by engineers claiming their application frameworks were complex, making them powerful. Looking at the code, seeing what the framework was doing, it was easy to realize that the frameworks weren’t complex…they were complicated. Twisty labyrinths of code that would leave you dizzy.

I was happy to see Jon Eaves post about complex vs. complicated, but I especially liked something I found in one of the comments:

the very definition of complexity is the emergent behaviour of simple pieces in interaction

You used to have to pay money for a service to render your website on multiple browsers and platforms to check for compatibility. Browsershots will do screenshots for free and is open source.

I’ve been tracking down a particularly insidious bug at work lately. Last night I had a breakthrough and discovered what I thought was a buffer overrun. This morning I built a test application that would trigger the overrun and ran it using Electric Fence. Let me say, thank you Bruce Perens.

Electric Fence takes over the memory allocation functions in C/C++. By default, for every memory allocation the application requests, Electric Fence allocates 2 memory sections: the memory segment that will be given to the application and another memory segment immediately after the first that is locked from being used by the application. Any attempt by the application to write into the second segment will result in (on FreeBSD, anyway) a SIGBUS. This is perfect for detecting buffer overruns because they are situations where the code writes past the area of memory allocated to them.

Needless to say, Electric Fence beared out exactly what I thought was happening. What a great tool.

I drove down today with the GPS hooked up to my laptop and NetStumbler running. NetStumbler is a WiFi detector application that runs under Windows. It takes your WiFi card and uses it to detect WiFi access points. If you have a GPS, NetStumbler will record the latitude and longitude of the access points it finds.

Once you’ve scanned the area you’re interested in, you can head over to GPS Visualizer. Upload your NetStumbler log file and GPS Visualizer will generate a SVG file showing all the access points you found. You can even select a mapping service to use for a map background. Once you’ve generated the SVG, GPS Visualizer will give you the option of generating a JPEG from it (see the image included in this post for an example).

In all, pretty sweet. I get a pretty good look at the 592 access points my computer saw along the way from Sacramento all the way down to San Jose. Access points with WEP or WPA encryption show up as red circles, open access points show up as green circles. Circles get larger for access points that have a stronger signal. If I hadn’t scanned such a large area, I could have used one of GPS Visualizer’s street-level map providers, but since I scanned a 140 mile path through California I had to use a less detailed map provider (I’ll probably split my log file up a bit later to generate more detailed maps of certain areas).

Update: I truncated the log file and regenerated the map, this time using one of the hi-res, street-level satellite images. See the shot of highway 50 west of Folsom. You can see one of Intel’s access points show up in the image. I should increase the background opacity, the maps aren’t very bright relative to the access point listings.

When I originally bought my laptop, one of the things I wanted to do with it was build a PC for my car. I didn’t really want the laptop to be the final product, but I wanted to be able to use the laptop to test things out, develop software and get things rolling before dropping some cash on the car-puter I really wanted.

My system consists of a couple of pieces:

• Computer: obviously I need to run all the software on something
• WiFi card or access point: I want to have network access in the car whenever there’s a usable WiFi signal in the area. I haven’t decided if I should do that through a card or a dedicated WiFi router. Either way, the setup will require a larger antenna than the default. I may be going as fast as 80mph in the truck at times, so extending the range of the WiFi connections it finds at that speed will be important. Also, being on the freeway, the extra range may help to pick up residential networks further off the road. And it may be a pipedream, but wouldn’t it be cool if nearby cars could connect to my access point during the trip? There could be some seriously cool caravan applications for that.
• Cell Modem: There are going to be times along the route where I don’t have a WiFi signal. Actually, trying to locate a WiFi connection, connect, get a DHCP lease and make any requests at 80mph may not be feasible. In those cases, the computer ought to be able to use Bluetooth to connect through my cell phone. It wouldn’t be high-speed, but I wouldn’t be looking to sneak out much more than a couple of packets every now and again.
• GPS: I’d like to know exactly where I am at all times. This comes in handy with the next piece.
• Local Search: Yahoo! has a local search service that will take the latitude and longitude as an argument. Wouldn’t it be cool to be connected via the WiFi and have coordinates via the GPS and be able to say, “where’s the nearest gas station?” Or, “where’s the nearest Burger King?”
• Yahoo! Music Engine: All the music in my collection available on the road. I’ve already been testing this out on the laptop using a cassette adapter and it’s awesome. Who needs the radio (local or satellite) when I’ve got all the music I could want at my fingertips? Also, with the YME plugins coming out now I could download audio/video news and traffic reports to the truck before pulling out of the garage.
• Voice Recognition: Being the driver, I really shouldn’t take my hands off the steering wheel and I shouldn’t be fiddling with a touchscreen or remote. Everything should be controllable through voice commands.

Those are the most important pieces to me. I’m sure I’ll think of more. Now I just have to get it all working, piece by piece. The hardest part will be getting all the software working together and provide an interface that’s easy to use while the truck is in motion.

Iain’s at the Ajax Summit today. I work with Iain and the other former-Oddpost guys at Yahoo!. They do some batshit crazy stuff with DHTML, so it’s no surprise to me that they’re representing at the Ajax Summit today.

I ran into FindBugs sometime earlier this week, but didn’t get a chance to check it out immediately. Tonight I played with it a little bit by turning it loose on the Java Yahoo! Search Web Services SDK. It’s neat, a very simple to use application that does some sanity checking on Java code and reports back any issues it finds. It found some data integrity issues (returning references to mutable objects and storing references to mutable objects) and some possible performance issues (inner classes that could be static inner classes). I think I need a much larger project in order to really put FindBugs through the paces, but this was enough of a test to determine that it’s very easy to use.

It comes with a GUI (pictured in the screenshot) and a command line interface. I used the GUI because, hey…why not? It has a nice tree control for expanding and collapsing the “bugs” it finds. Additionally, a tabbed interface allows viewing the “bugs” by different groupings (by class, package, bug type and bug category). If you get a chance to point it at a larger project, let me know how it goes. I’m especially curious to see how it does against codebases that have glaring bugs.

I’m doing some sample code for talking to the Yahoo! Search Web Services. I’m writing it in Java and figured I might as well take the opportunity to write it using the latest and greatest: J2SE 5.0 (or is it 1.5? or is it Tiger?). One nice thing about J2SE 5.0 is the new XPath support. I’ve always felt that SAX is too low level an API for directly dealing with XML and DOM is just too much of a pain in the butt because of all the code you have to write to traverse down to a particular element.

The XML I’m attempting to parse is a simple web search for Ryan Kennedy. I utilized the javax.xml.xpath.XPath object and the evaluate() functions it provides. Notice, there’s two ways of getting XML into the XPath evaluation() method: using an org.xml.sax.InputSource or using a java.lang.Object.

I tried using an InputSource first, since it meant I didn’t have to do anything other than create an InputSource instance that wrapped a java.io.InputStream. This presented an initial problem. When I tried evaluating the XPath expression “/ResultSet/Result/Url”, I should have gotten back a org.w3c.dom.NodeList instance. But I didn’t. If you look closely at the XML, you can see there’s a default namespace configured: xmlns=”urn:yahoo:srch”. I had to change the XPath expression to be “/urn:yahoo:srch:ResultSet/urn:yahoo:srch:Result/urn:yahoo:srch:Url”. This was ugly as hell and not intuitive at all, but it worked. I decided I wanted to also print out the totalResultsReturned attribute from the ResultSet element, so I made a second call to evaluate(). This caused the second call to fail. Evidently, when using an InputSource you can only evaluate once.

So I switched from using the InputSource version of evaluate() to the Object version of evaluate(). It turns out you can pass in a org.w3c.dom.Document object in. So I used the javax.xml.parsers.DocumentBuilder to construct a DOM document object from the XML. I passed the Document into evaulate() and didn’t get anything back. I stripped off the namespace prefixes on my XPath expressions and it started working.

So, if I use an InputSource with XPath.evaluate() I have to concern myself with namespaces. If I use a Document with XPath.evaluate() I don’t. Maybe I’m missing something, but the behavior of the XPath support doesn’t seem to be very consistent.

If it’s true that developers are flocking back to C++ from languages like Java and C#, maybe I don’t want to be in this profession any more. I used Java for 4 years after college working for 3 different internet companies. I’m now at my 4th internet company (Yahoo) doing C++ development and I must say, it’s the most excruciating change ever. Give me back my bounds checking. Get rid of my seperate specification and implementation files. For god’s sake, TAKE THE GODDAMN PREPROCESSOR AWAY FROM MY COWORKERS.

“Maybe I could learn how to be a truck driver. Mav, you still got that number for that truck driving school we saw on TV? Truckmaster I think it is…I might need that”-Goose

“I feel like I’m taking crazy pills!” -Mugatu

These instructions on docking the JavaScript console in the sidebar might be useful to some people. I use the console quite a bit and having to access it through the “Tools” menu can be a pain (there’s no configured hot-key for it either). The console, when placed in the sidebar, is always visible and available. Using the console from the “Tools” menu makes it a seperate window (I find that to be a little clunky when I’m alt-tab’ing between an editor and Firefox).

I decided to finally download Java 5 tonight. I want to play with the new concurrency support. While I was downloading, I found a bunch of articles linked off of the pages. One thing that caught my eye was Jester.

Jester is a neat idea. You have a bunch of code and you have a bunch of unit tests. The unit tests check to make sure your code doesn’t have any bugs. But how do you know if your unit tests have bugs? How do you know if your unit tests adequately check for bugs in your code? I’ve used code coverage tools in the past (i.e. Clover) and they’re very cool for telling you whether or not your unit tests adequately cover your existing code. Jester takes things a step further. It actually takes your code and tweaks it in subtle ways, trying to create bugs. It then runs your unit tests and checks to see if your unit tests picked up on the bugs it introduced. If not, Jester concludes (sometimes incorrectly) that your unit tests have bugs.

Maybe it’s just me not paying enough attention, but I never hear about projects like that for other languages. Maybe I’m just more hooked into the Java world than I am with any other language, but it always seems to me like some of the coolest tools for creating quality projects are built for Java.

I’ll totally welcome being proven wrong on that last point.

Ken “I’m crazy MOO man” Fox got a chance to play with the Yahoo! Search Web Services last night. He brings up a very good point about how easy it is to get started. Even without using one of the provided SDKs, the REST model used by Yahoo! makes it dead simple to use the service. If you can request a URL and parse XML, you can use the service.

Contrast this with the Google search web services. You either have to know how to POST and process SOAP envelopes or you have to pull in an entire SOAP framework. The Google service was cool when it came out because it was new. Stacked alongside the new Yahoo! offering, it pales in comparison.

Of course, the real indication of who’s service is better will be determined by the applications that come out.

I got to write the Java implementation of the Yahoo! Search Web Service SDK. Today I set up a SourceForge project where I will maintain that software package. This is the first piece of software I’ve released publicly. Up until now my software has only seen the light of day in the form of services offered by other companies (most notably, Excite Inbox and EarthLink Webmail). I really enjoyed building the SDK and am looking forward to whatever piece of software I decide to release next.

Yahoo! Search blog: Announcing the Yahoo! Search Developer Network and Search Web Services

Yahoo’s finally announced it, so I can too. The Y! Search team released their search web services tonight. I’ve been playing with the services internally for some time now. I even contributed to the SDK. So have a look, play around and make something cool so we can help you pimp it out to the rest of the world.

I’m doing some work on a side project within Yahoo!. Among other things, the code needs to be able to make HTTP requests. Using java.net.URL isn’t very tantalizing, mostly because it’s a total pain in the ass (try getting the returned content when an HTTP 400 error code is sent back to you, try building up a query string).

One of the only other Java-based HTTP libraries is available as part of Jakarta: HttpClient. So you download the JAR, dump it into your project, compile and run. BOOM! It blows up in your face:

    java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory

Okay, you need to have the commons logging package in order to be able to use HttpClient. I really hate the whole idea behind the commons logging package, but I need HttpClient. So I (grudgingly) pull it down and add it to my project libraries. Run my project again. BOOM! It blows up in your face:

    java.lang.NoClassDefFoundError: org/apache/commons/codec/DecoderException

What the?!? What the hell is commons codec? You have to be kidding me. I have to pull down ANOTHER dependency? I was hoping to keep dependencies to a minimum with this project. Now I have 3 brand new dependencies just because I wanted to make some stupid HTTP requests.

I guess the moral of the story is, if you want to keep dependencies to a minimum you might as well handroll the functionality you want.

As a side note, those 3 new dependencies are 350KB in size. So much for making something lean that could possibly be used on an embedded device.