Thoughts on The Future
Making Erlang indentation-sensitive
Ulf Wiger has been experimenting with making Erlang indentation-senstive. He followed up with a Part 2 after comments.
For most people who start working in the Erlang language, the most annoying new concept is all the various line-terminators you have to know. Damien Katz sums this up nicely. All these explicit terminators might make other people's code easier to read but writing code should be easy too (they are equally important). Mue says just get over it but as a Python user I agree that Erlang would be way better off if it was indentation-based so this was an interesting experiment.
As for Python, I'm still shocked to hear people say "Python would be nice ... if it wasn't for that whitespace thing." Who is perpetuating this? I have no idea. The next time you hear someone say this, pass along Python: Myths about Indentation!
- 3 comments
- read comments
- post comment
- permalink
PyPi (Cheeseshop) on Google App Engine
Like many of you, I've had my jaw on the floor since the release of Google App Engine. Although there are skeptics out there, a careful read of their terms will show you that it's for real — Google has released GOOGLE to the world and it's not for scary marketing purposes. In fact, I've been growing tired of paranoid Google haters; I'm hoping this will shut them up for a while.
Why is App Engine such a breakthrough? The concept of a hosted web application is nothing new but it has never been done this well. Mundane server maintenance? Gone. Infinite scalability? Check. 100% uptime? Let's face it, if Google went offline you'd probably be down in a nuclear bunker playing Parcheesi.
So ... how should we leverage this tool for the greater good of the community? I can't count the ways without getting dizzy. How about let's start with a mirror of PyPi, the Python Package Index?
PyPi on the App Engine
I barely spent two days on it, but here it is: http://pypi.appspot.com/. Test it out, play with it, try to break it.
As Python grows, especially due to App Engine, PyPi needs to scale too. Zope has put together a PyPi mirror but that's the only other one I know of (actually, I can't even find the link to it right now). Coincidentally, PyPi even went offline for a few min while I was writing this blog post.
Issues...
- urlfetch can't get big packages
- fetching SQLAlchemy triggered the ResponseTooLarge error but its package was barely over 1MB. We might be able to create a download proxy to work around this.
- May not be fully compatible with easy_install yet
- Can't run scripts on App Engine
- Not too big a deal, you have to think of web pages as scripts and use a separate crontab to make timed requests. There are security issues with this, but for mirroring pypi it's not a problem
- Note that I haven't actually mirrored all of PyPi, I'm still testing it out.
- It's not very pretty yet (see below)
- The prototype works, now it needs some tests :)
You Can Help
I'm not dedicated to this project, I just thought it sounded like a good idea and would be a fun way to experiment with the App Engine. If anyone is interested in working on it just let me know --kumar.mcmillan@gmail.com. If there is enough interest I'll put it on Google Code. Possibly the most exciting feature of App Engine is the Datastore API (aka BigTable) and Ben Bangert agrees. It's a little hard for me to wrap my head around it but so far the Expando class—besides being the coolest name for a class—seems to work great for storing package data. If EGG-INO grows a new parameter, it just gets tacked on to the row dynamically.
This has also been a great way to dig up bugs, some of which have already been fixed.
- 6 comments
- read comments
- post comment
- permalink
Software is written by hand
...that's right, it's not molded or prefabbed, it's not made on a production line or in a lab. Are we insane??! Here is a hilarious probe into the darker side of this art we call programming.
- 0 comments
- read comments
- post comment
- permalink
How To Get Started Writing Open Social Applications
I started learning about Google's Open Social API and gave a lightning talk on it at the last Chicago Python User Group meeting. It wasn't much more than a brief overview and fodder for "diving in" to the Open Social platform. Here it is again, in more of a tutorial form, so dive in!
What Is It?
It's a set of programming interfaces to let you build "applications" that run inside social networking websites. By applications, think "widgets." The interfaces consist of a JavaScript API to build the screens and interactivity and a REST API (Atom publishing API + XML) for the "container" site to implement.
The official "container" sites so far include Friendster, MySpace, Orkut (Google's own), Hi5, Ning, and several others. Some interesting facts from the video : Orkut has an overwhelming presence in Brazil and India and Hi5 is the largest Spanish-speaking social site. Ning is unique in that it's a "meta" social site. It's a social site that enables you to build your own social sites until your head explodes.
The Role of An Open Social Application
I predict that the dark energy of the universe (that is, energy generated by the free time of cunning web developers like you and I) will bring hundreds of cool social apps into existence. It's the same collaboration concept that makes open source software so successful. And more similarly, what makes Firefox so customizable through Extensions. This kind of thing gets me excited about the Web reminiscent of when RSS became a standard. To be honest, I don't think social networking is that interesting and I am still waiting for the fad to die out, but Open Social might encourage other sites to standardize additional container/application APIs.
Let's See Some Code
To test out an Open Social app you can use the Orkut Sandbox. It needs to conform to the Google Gadget API; here is the simplest app you can build:
<?xml version="1.0" encoding="UTF-8" ?>
<Module>
<ModulePrefs title="Boring Hello App">
<Require feature="opensocial-0.5"/>
</ModulePrefs>
<Content type="html">
<![CDATA[
<script type="text/javascript">
function start_hello() {
document.getElementById('message').innerHTML = "Hello!";
}
_IG_RegisterOnloadHandler(start_hello);
</script>
<div id="message"> </div>
]]>
</Content>
</Module>Since this isn't very exciting and doesn't show off and of the API features, there are plenty more examples in the Open Social docs.
Some Gotchas
The Orkut sandbox has a fierce cache. To use your application uncached, you have to add this on to the end of your app's URL:
http://.../?...&bpc=1Since the app is just HTML, you can use the script tag to pull in 3rd party JavaScript libraries. But keep in mind you will face some problems due to to JavaScript's Same Origin security policy. Although, the JavaScript libs automatically available will probably let you do most of the remote-call things you need, like fetching JSON. There also might be some workarounds with the Dojo framework's iframe strategy.
Have fun!
- 1 comment
- read comments
- post comment
- permalink
GTAC Highlights Part 1 - Selenium is Alive and Well, Model Based Testing Is Smart, And...
I just got back from the GTAC (Google Test Automation Conference) in New York and had a great time. It spanned 2 days and had a single track — this made it very laid back (no headaches trying to decide what talk to attend) and the timing was perfect. Especially since my traveling managed to dodge one of the worst summer storm systems to hit Chicago in at least a decade!.
I've put together some highlights using the notes I took at each talk. Please bear in mind that this is not a comprehensive report on the conference and may contain misinformation (feel free to comment with corrections). The Google folk did an impressive job of posting video of most talks online within hours. A youtube search for GTAC lists them all. Or ... you can watch them from a playlist
Allen Hutchison - First Principles
- Youtube video
- There were about 300 applicants and only 150 were accepted. We had to apply with a short essay about why we should attend. This sounds elitist but actually it made it so the conference was full of people who really wanted to be there, which is very cool. Oh, and it was free so I guess something like this was necessary.
- Allen mentions a test framework he has been working on, google-oaf (Google Open Automation Framework). If I heard correctly, this was used for testing Google Code?
- A little about the Google Test Engineering dept.: There are teams of dedicated "Test Mercenaries" who will refactor a newish project's tests and hand them back.
- Assures the audience that any demo to fail means it's on the cutting edge!
Patrick Copeland - Keynote
- Youtube video
- Now works on the Test Engineering group at Google.
- Talked about how his team brought the build/test process for iGoogle down from 77 min to 14 min.
- A bit about using mock objects to simulate faults - the only way.
- Concept of "Happy Path" tests — ones that are designed to pass for the way a user should be using the application.
- Referenced James Reason's Swiss Cheese Model to talk about how interactive components can be full of undiscovered holes since the possibilities for their interaction are endless (again, talking about mock objects).
- Google practices "Root Cause Analysis." That is, instead of a team getting stuck on a treadmill with workarounds they are given extra time and resources to fix the actual problem. This sounds like a no-brainer but I see the treadmill happen all the time. Some problems are really really hard to track down and the business doesn't have time to slow down while the proper probing is done so that they're fixed. When I pressed him more in Q&A it was more like what I thought: a trade-off / balancing act. Still, great to here that from a high level their business is allowing this kind of "Do The Right Thing" to happen.
- Talked about Google's Selenium farm (huge grid of machines to run Selenium tests).
- Mentioned Eggplant, GUI tool for running Selenium tests?
Simon Stewart - Web Driver for Java
- Youtube video
- WebDriver is a Java library that can drive a real browser (currently Firefox, IE) or a fast in-memory implementation of one.
- Proves that web driver (or Locomotive?) is on the cutting edge! The demo failed due to the fussy Locomotive app (Rails on OS X)
- The demo wasn't really necessary though since the code did the talking. This is a very well architected library for web testing.
- Especially interesting is how it suggests creating Page objects for each HTTP response. This objectifies all the page elements and makes for a more maintainable test suite since details that change often (click twice, toggle button "foo," etc.) can be declared separate from the test / assertion code. Hmm... I might steal this idea when I next write some twill tests ;)
Ryan Gerard and Ramya Venkataramu on Test Hygiene
- Youtube video
- Test Hygiene (is that the name?) is an in-house application written for their company that provides an interface to rate the effectiveness of tests for various products.
- Pretty interesting idea: you have a community of developers who run each others tests and rate them on things like documentation level, sanity, effectiveness (are they testing the right thing?), and some other subjective qualities.
- Keeping tests maintained and in good health is way harder than doing the same for production code.
- Most of the Q&A seemed concerned with "gaming" the system, this didn't seem worthy of such lengthy discussion, IMHO.
- One thing pointed out by Q&A is that the system doesn't seem to integrate well with historic code revisions. I think this will be a challenging feature to add.
- Referenced James Surowiecki's The Wisdom Of Crowds
- Talked about "easy grading system" (i.e. eBay?) — can someone explain to me what this means?
Matt Heusser & Sean McMillan - Interaction Based Testing
- Youtube video
- The "balanced breakfast:" a combination of mock objects for isolation and functional tests against real objects for interaction.
- Everyone at the conference seemed to have a love/hate relationship with mock objects.
- This talk made me want to use mock objects a little more (not a lot more!), perhaps because I don't really use them at all ;) They are definitely useful for deducing the root cause of a failing functional test.
- One interesting idea they suggested was creating "facades" (groups of objects) so that a mock facade could be installed for a test. This suggests the idea of "switchable" mock objects — i.e. an "on" switch to make tests run faster, an off switch to make the tests run better, but this wasn't addressed in the talk and I forgot to comment on it.
- They also talked about one possible strategy of focusing on "negative" testing in unit tests since functional tests naturally test for positive use cases. For example, one could just unit test a login method for error cases since all other functional tests would need to login successfully to run.
- The Mars Rover bug: two different "units" failed to operate together correctly because one operated in English units and the other in metric units. Again, why mock objects alone aren't good enough!
- Sean and I chatted later in the hotel lobby about the McMillan Clan Scottish tartan :)
Adam Porter & Atif Memon - Skoll DCQAS
- Youtube video
- This was by far the most engaging talk, I highly recommend watching the video. Actually, it was really two talks in one, so I broke it down accordingly.
- Skoll DCQAS stands for Distributed Continuous Quality Assurance System.
- Adam posted this link but the Skoll system isn't officially released yet. That link is to a package that supports Skoll to test the MySQL database product. Hoping to release an official Skoll package soon.
-
Adam's talk
- This is the problem: You have a software product that can be compiled on many different OS platforms and has many different configuration options. How do you test all possible combinations!
- Skoll is used to build a matrix of all these combinations then manage a distributed farm of servers that run the product's test suite in each configuration/platform. This runs continuously, triggered by each revision made to the code.
- In the first project he implemented this with (some CORBA system), it would have taken something like a full year to run the test suite once per each configuration combination. Instead, Skoll makes a map of all combinations in relation to one another and randomly picks points that are far away from each other. When one point fails, it starts digging further by testing the closest points.
- Similar thing for the MySQL product: there are 110,000 possible configuration combinations for installing MySQL.
-
Atif's talk
- Atif was using Skoll to analyze GUI applications and automatically generate test cases based on all the possible interactions of the GUI widgets.
- This system is called GUITAR and is available for use.
- An interesting thing was discovered: when you look at all the possible paths a user can take through GUI widgets, often the shortest path is that which is most used by real users. This makes a generated test case for such a path very useful.
- Atif chose 4 popular sourforge apps and ran it through the system. This resulted in a handful of bugs discovered for each app!
- I'm very taken by this approach and may attempt something similar for AJAX web applications where widgets interact with each other in complex ways. However, it will probably be difficult to introspect common inputs/output of Javascript objects — they may have to be declared.
Apple Chow & Santiago Etchebehere - Building an Automated Framework Around Selenium
- Youtube video
- The framework is called "Ringo" because their code is "...getting better all the time..." :)
- Interacts with Selenium RC via Java
- Why are Java developers so enamored with XML? All the declarative data structures used to wrap up Javascript objects are done with XML. This seems to me complete overkill and would be such a pain to edit all the time.
- Very cool idea: UI objects are represented in test code as objects too so that the implementation for testing them (click button X, wait 2 seconds, etc) can be hidden. Much like Web Driver's approach this makes it so you don't have to alter test code when UI implementation changes.
- Developed for Top Secret Google systems but showed an example of how one could test the Google Suggest interface. Showed how the search-as-you-type object would have a custom implementation for how it can be tested — loops through the text field, sends each character to Selenium RC, waits a second, sends another.
Doug Sellers - CustomInk Domain Specific Language for automating an AJAX based application
- Youtube video is not up yet at the time of this writing.
- Also a very excellent presentation, keep an eye out for this video.
- Talks about writing a DSL that runs in Ruby and Selenium on Rails (not using RC, actually compiles HTML to run in Selenium)
- This was built to test Custom Ink, a site that has a Javascript (ok, ok, AJAX) interface for customizing a t-shirt order — colors, typeface, graphics — before submitting.
- The DSL is built specifically for the site, I.E. "add_graphic" might be an action. There are 60-80 commands. Goes so far as to say click_link "browse gallery" instead of hard coding div IDs / xpaths, this is pretty smart.
- Sort of makes me want to try this in twill, that is, build a higher level language that boils down to click/wait twill commands in the background.
- Allows again a good separation of test case logic and test implementation, the click/wait interacting with the interface.
- Empowers developers who are not good with Javascript to write functional tests for the website.
- Designed for business users to write tests? Not so much. This came up during Q&A.
- This was addressed a little in Q&A but not completely: I wonder how helpful and/or easy to pinpoint failures in a test case are. I imagine "order page has no button named submit" would induce a lot of head scratching. But, alas, this is always a problem with functional testing.
- Advice: Use CSS selectors, not xpath in Selenium, as it is way too slow in IE.
- Can't test file uploads with Selenium except in Firefox.
Risto Kumplainin - Automated testing for F-Secure's Linux/UNIX Anti-Virus products
- Youtube video is not up yet at the time of this writing.
- They built a cool LED panel for their office that summarizes any failing tests in a grid of OS/configuration.
- "A product of evolution, not intelligent design" — this might just be the quote of the conference.
- One module named "moosetest" after the Saab crash test that simulates what happens when a car hits a moose!
- Typical success story for automating a continuous build/test process when many combinations of configuration exist for the product.
Jennifer Bevan & Jason Huggins - Extending Selenium With Grid Computing
- Youtube video
- Jennifer has worked on Canyon (sp?), an open source data mining tool. This sounded interesting. Does anyone have any info about this? Google wasn't very helpful. And she works there! Sheesh.
- Everyone complains that Selenium tests are slow but this is a constraint of the brower itself (duh). I also find it funny that everyone complains about this.
- Used to test the Gmail UI at Google.
- Runs python test code via Selenium RC in parallel against Firefox/IE in multiple machines. This greatly speeds up test execution time :)
- A live demo of using Amazon's EC2 (Elastic Computing Cloud) to run tests in parallel like this. And it worked!
- Can run multiple instances of Firefox (via user profiles) on a single machine. Cannot run multiple instances of IE on a single machine.
- Chrome for Firefox bypasses common security issues, IEHTA for IE does it similarly. None of these are used in the Selenium implementation (yet?).
- Ran into many issues where Selenium RC would deadlock over time, leak memory, browsers would time out unexpectedly, etc. Workaround right now is to restart the worker machine or search and destroy while looking into a fix. What's up with that Patrick?? What about Google's Root Cause Analysis? :)
- This talk was great news for Selenium, I remember when using RC was a painful, scary endeavor due to its instability. Also, the thought of running over 200 Selenium tests made one think of comics like this (it is very slow).
That's all I have time for at the moment. Check back for Part 2 - coming soon!
- 9 comments
- read comments
- post comment
- permalink
Humans are here to stay!
Slashdotters are getting bummed out that Amazon is trying to patent its Hybrid Machine/Human Computing Arrangement. The bitterness seems to be in the gist of "computers are taking over the world, oh no!" or something.
As some comments note, this is a logical follow up to the Amazon Mechanical Turk, a web service of sorts that provides real human intelligence as a "processing node." I know a lot of people like to rant about the impending apocalypse but I think what Amazon are doing is really smart. They are basically saying, look, there are a lot of problems that are hard for computers to solve but trivial for humans to solve. Say, for example, given 50 images, find how many of them have a Dalmation puppy in them. That's hard for a computer but takes a real person milliseconds.
This may not be a future proof example, as this visual cortex simulator idea suggests, but let's face it, computers are not the answer to all our problems and they never will be, no matter how much they advance. We will always need humans! In fact, desktop computers suck so bad we have a long way to go before they are even understandable by the majority of people. Sorry, I just can't get on this doomsday view about how robots are going to take over our lives. I think, if anything, Amazon is setting a very positive trend for the future.
- 0 comments
- read comments
- post comment
- permalink
Why People Don't Use Hand Dryers
It's because they don't know that you have to 1) wring out or shake off extra water then 2) rub your hands vigorously under the dryer. If you don't do that, it doesn't work (unless it's some turbo dryer I've seen in restaurants recently). Most dryers do not have these instructions printed on the dryer. Are people stupid? I won't answer that, but let's be fair, it's not very intuitive. Tools need to be intuitive and dead simple to succeed in today's society. This is why a lot of people curse computers and/or hate certain desktop apps.
Having said all this I was a little surprised to hear that Humanized—a Windows application [1] that works like Quicksilver in that you can just start typing something you want to do—is gaining popularity. I was surprised because I thought mainly programmers thought "with their fingers" since they [at least I...] find 10 fingers to be some of the most useful tools on the body. However, it also doesn't surprise me because this is a very intuitive way to work: start typing (or, possibly, start saying) what you want to do ... then watch it happen.
Does this mean we are about to see a counter-revolution to the mouse? I certainly hope so.
[1] behind the scenes I believe Humanized is written in Python and Freetype, but that's not so relevant.
- 1 comment
- read comments
- post comment
- permalink
