“Everything changed,” our project lead replied, “and the documentation is pretty poor.”

In general, one of the characteristics of a good API is stability. Once published, partners in the outside world build code which depend intimately on the API spec. Well-designed APIs evolve slowly, adding and removing features gracefully so partners can adjust their apps to the improvements.

MSN jerked the rudder hard in a new direction going to v 5.1. Below is a list of pitfalls our engineers encountered. Hopefully this list will help others struggling with MSN’s docs.

Beyond MSN’s business challenges competing again Google and Yahoo, this list demonstrates some of the technical challenges MSN is facing with their codebase. To their credit, the MSN API team has responded quickly and knowledgeably to our requests. For that we are grateful. We appreciate the technical help we receive from MSN, and we hope their API stabilizes relatively quickly.

If any MSN folks are reading this, if it would be helpful, our project lead on this recent API upgrade would be glad to share his experiences with you folks on how in the future this could go more smoothly for both sides.

For contrast, about the same time period, our engineers upgraded our platform from Google API v11 to v12.

The Google API upgrade took 20 minutes.

To be fair, Google v11 to v12 was an extremely small and gradual API tweak, a handful of type changes and other little details. Comparing the Google 20 minutes to MSN’s many developer-days of effort is comparing apples to oranges. I would say, though, that Google’s API advances more gradually and more thoughtfully, making Google an easier technical partner than the other engines.

Again, our thanks to the MSN engineers for fielding so many of calls and emails to clarify the docs. The following email snippet indicates some of the places our engineers hit snags.

There there were more of these, but here is a brief list. In overall they changed the external interface, including names, enumerations and methods. This was more like a full rewrite than an upgrade. My biggest negative on the list is the documentation, as this was the cause of much of our frustrations.

And to give credit - MSN responded quickly and knowledgeably to our requests.

• .Net2/asmx (Administration, CustomerManagement, NotificationManagement interfaces) .Net3 (or 3.5?)/wcf (CampaignManagement, Reporting interfaces)
  • changed xml structure (authentication, etc)
  • enforced xml sequence
  • appears to be that the upgrade is not full, likely an internal deadline was not met
• Renamed many fields and classes to comply with industry standards
  • API is for machine communication, so the name only matters to developers, who has to redo the code now
• Changed method enumerations
  • most of these are 1-to-1 renaming enums, what is the point?
  • eg. removing underscores from EasternTime_US_Canada
• Xml arrays for NegativeKeywords (yes, no longer notkeywords)
  • fine, although web-interface uses comma separated list
• Date type of fields
  • communicate as 3 xml tags: year, month, day
  • link to docs
    • looks like a bug workaround, as field supposed to be nullable!
• Inherited classes (eg: Ad to TextAd and MobileAd)
  • readonly “Type” field is set by derived class type
  • http://msdn.microsoft.com/en-us/library/bb671952.aspx
  • creating ads as Ad xml tag with xml attribute of TextAd, but eliminate Type field during soap operations, as it is readonly…
• API DOCUMENTATION
  • bad navigation (needs to click through many pages to get to the right page, unless you know what you need, and use the sidebar then)
  • INCORRECT information
    • Campaign BudgetLimitType field for a while listed v4 enumerations! - incorrect hyperlink (simple MSN internal curl could discover these after removing the old docs from their internal website, like what we do with our website :-))
    • Parameter substitution in some Ad fields were listed without substituion (appears to be fixed by now)
  • offline version for windows framework only
  • xml code snippets are limited and started to appear in June (middle of the upgrade process)
• Why to have fields, which are reserved for future use? (like v4 had APIFlags, which never made to production, and now is eliminated)
  • Nice to see the ideas, where MSN might go
  • But has no value for an API, and makes documentation readability worse as it litters the information with useless stuff
• V5 was retired ~2 months before deadline, and 5.1 took its place (5.1 had some new reserved for future use fields, AND expanding int to long fields)
  • not a big deal, but what is the point to publish deadlines?

 

RKGDuck slides a snippet of perl (or a big honking mass of perl, should you so need) into the Windows cut-and-paste clipboard buffer. This lets you do powerful text transformations within and between all Windows apps, using the familiar Control-C (copy) and Control-V (paste) mechanism.

After cajoling your local perl geek to write a tiny filter for your task, you can share that functionality easily with less-technical users, and you don’t have to install perl on the end-user machines.

Here’s a new RKGDuck video showing how in a couple minutes you can write a powerful filter to help clean up keyword lists.

You can watch other how-to videos and download the app itself on the RkGDuck homepage.

RKG provides RKGDuck free to the community under GNU General Public License. RKGDuck us distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantibility or fitness for a particular purpose.

If you’ve found RKGDuck helpful and can share your experience, or if you’ve hit any snags, do comment below.

Quack!

YAPC::NA 2008 was a wonderful geek conference which occurred last week in Chicago. RKG strongly supports the open-source movement, and we do our best to give something back to the community through sponsorships and code.

Kudos to the other fine organizations who chipped in:

Google, O’Reilly, ITA Software, Summersault, Grant Street Group, Deutsche Börse Group, CheetahMail, Rakudo, OpenMake, Rimm-Kaufman Group, Active State, Knowmad, WhitePages, Apress , PackT Publishing, Pragmatic Bookshelf, No Starch Press, PlainBlack, CarSpot, Where2GetIt, VirtualMin, Allant, CBS Sports, DRW Trading, Free Software Magazine

The odds are much lower that you’re toiling in a three-person web 2.0 startup.

Nonetheless, there’s considerable wisdom to be gained from 37Signals Getting Real book. This series of essays focuses on what makes 37Signals successful.

With some translation, many of their points are relevant to online retailers. Yes, in Marketing, you probably have limited control over your developers, and yes, as a complex retailer, simplicity went out the window several years back. Regardless.

Still. I recommend checking out Getting Real, and seeing which of their suggestions could help make your company more nimble.

Assorted quotes I bookmarked:

Constraints are often advantages in disguise.

Most of the time you spend is wasted on things that just don’t matter. If you can cut out the work and thinking that just don’t matter, you’ll achieve productivity you’ve never imagined.

Estimates that stretch into weeks or months are fantasies. The truth is you just don’t know what’s going to happen that far in advance. So shrink your time. Keep breaking down timeframes into smaller chunks. Instead of a 12 week project, think of it as 12 weeklong projects.

Set up a rule at work: Make half the day alone time. From 10am-2pm, no one can talk to one another (except during lunch). Or make the first or the last half of the day the alone time period. Just make sure this period is contiguous in order to avoid productivity-killing interruptions.

Do you really need a meeting? Meetings usually arise when a concept isn’t clear enough. Instead of resorting to a meeting, try to simplify the concept so you can discuss it quickly via email or im or Campfire. The goal is to avoid meetings. Every minute you avoid spending in a meeting is a minute you can get real work done instead.

If you are trying to decide between a few people to fill a position, always hire the better writer. It doesn’t matter if that person is a designer, programmer, marketer, salesperson, or whatever, the writing skills will pay off. Effective, concise writing and editing leads to effective, concise code, design, emails, instant messages, and more.

Epicenter design focuses on the true essence of the page — the epicenter — and then builds outward. This means that, at the start, you ignore the extremities: the navigation/tabs, footer, colors, sidebar, logo, etc. Instead, you start at the epicenter and design the most important piece of content first.

That’s why context is more important than consistency. It’s ok to be inconsistent if your design makes more sense that way. Give people just what matters. Give them what they need when they need it and get rid of what they don’t. It’s better to be right than to be consistent.

Your code can guide you to fixes that are cheap and light. Pay attention when an easy path emerges. Sure, the feature that’s easy to make might not be exactly the same as the feature you originally had in mind but so what? If it works well enough and gives you more time to work on something else, it’s a keeper.

Build, don’t write. If you need to explain something, try mocking it up and prototyping it rather than writing a longwinded document. An actual interface or prototype is on its way to becoming a real product. A piece of paper, on the other hand, is only on its way to the garbage can.

Lorem ipsum changes the way copy is viewed. It reduces text-based content to a visual design element — a shape of text — instead of what it should be: valuable information someone is going to have to enter and/or read. Dummy text means you won’t see the inevitable variations that show up once real information is entered. It means you won’t know what it’s like to fill out forms on your site. Dummy text is a veil between you and reality.

Link: Getting Real, 37Signals

The cart was dynamic on the old site, of course, but their huge number of SKU pages weren’t. SKU pages were static HTML, generated by scripts each night.

This meant big headaches for their merchants. Think mid-day price changes (disallowed!) and mid-day stock-outs (site kept selling sold-out SKUs!)

So they switched onto a (then) modern e-comm platform.

The new site solved the mid-day merch change issues, but came with some downsides: ugly SEO unfriendly URLs with sessionids, a much slower site, increased hardware costs, increased platform costs. And they didn’t remap the old URLs properly, so they lost a great deal of SEO-driven revenue. Ouch.

This experience got me thinking about how little on a e-comm site is truly dynamic intra-day.

Prices and quantities, yes. In-stock and out-of-stock, absolutely. Perhaps user reviews and ratings, but those don’t need to be true real-time.

(Completely different from a news or sports site, where everything must be up-to-the-minute fresh.)

So, here’s a crazy idea for a e-comm architecture:

Render the whole site in static HTML each evening, with prices and quantities accurate as of when written. Then use tiny fast AJAX calls throughout the day to update only those few bytes on each page that actually change.

Just a thought.

Open source code is typically high-quality, lets you avoid licensing fees, and most important, you can lift the hood and fix any bits that come loose.

RKG is among the many many companies that owe a large debt of thanks to the open source movement.

But, of the many small businesses using open source, only a small fraction give back to the community.

One way to support open source is donate to the organizations and conferences.

Again this year, RKG is making a donation to help the good folks at YAPC::NA2008, and urge others using perl to do the same.

Another way is to contribute code.

Bug fixes. New features. New applications. What have you.

If your company uses open source, and you improve the code, seriously consider sharing it back.

While we don’t release the proprietary code which provides RKG and our clients with unique technological and/or business advantages, we do share back “housekeeping” code.

For example, for both RKG and for the community, Baron Schwartz developed table archiving nibblers, table synchronization helpers, server monitoring scripts, and other useful mysql tools. He bundled these apps as MaatKit and released them under the GNU General Public License.

What a honor for Baron that last week at the MySql conference he was given a “MySql Community Member Of The Year” award. Kudos, Baron!

It doesn’t matter if your donation is large or small. It doesn’t matter if you give money or code.

What does matter is this: if you’re benefiting from the Open Source Movement, try to give something back.

It makes good business sense. And it is the right thing to do.

Quoting Google,

Google App Engine enables developers to build web applications on the same scalable systems that power our own applications.

No assembly required: Google App Engine exposes a fully-integrated development environment.

It’s easy to scale: Google App Engine makes it easy to design scalable applications that grow from one to millions of users without infrastructure headaches.

It’s free to get started: Every Google App Engine application will have enough CPU, bandwidth, and storage to serve around 5 million monthly pageviews.

Right now the only dev language is Python (snakes on a plane!), but others are promised.

The API currently provides storage (vs BigTable), authentication (via Google Accounts), templating (via Django), emailing, form handling, and URL fetching. Everything runs in the cloud, sandboxed, no writing to local files.

Here’s Google’s Brett Slakin demoing a guestbook in 10 minutes.

And here’s the resulting guestbook app: http://shoutout.appspot.com/.

Relevance to online retailers?

I don’t think AppEngine is going to change or revolutionize online commerce as a platform.

The hard problem in e-commerce isn’t scaling the technology, the hard problem is marketing, getting the customers.

But I do think AppEngine (and EC2, etc) could revolutionize the speed which new Web 2.0 communication apps are built.

In the next year or so, that may have implications for how online marketers communicate with customers and prospects.

My thoughts after reading the AppEngine docs was:

“Wow. With this tool, a smart small agile development team of three people could beta something like Twitter in a few days, or maybe Meebo in a few weeks, and release it, and if the world liked it, they could scale their app up to the planet, GoogleSizing it, with zero capex, zero sysadmin and zero hardware hassles, and simply pay for cycles/bandwidth/storage as it scaled, as it proved itself viable via revenue or pageviews or buzz or whatever. You’d build a layer between the code and the API to facilitate moving the app later to dedicated servers as needed if concerned about autonomy, but AppEngine slashes the VC needs of a startup even more — now the costs are really just people, no machines — and handles the scaling problem pretty far along the way.”

(I also thought, “Dang — I wish I had more depth in Python!”)

The apps available now are demos and toys, but more will come.

I guess the entrepreneur / developer considering AppEngine need to answer the “how much do I trust Google to park my app on it”, but I think that’s less of an issue for the freshly minted startup.

My prediction: cloud app platforms will increase the velocity of experimentation and deployment of new crazy marketing and communication ideas.

Overheard at Shop.orgSocialMedia2010:

What, you’re not advertising to teens via TwitzaZooMeeBling yet?!? You’re so far behind the times. They’re 90 days old but they’ve already hit 12 million registered users! And the only hardware they own are the founder’s laptops!

If anyone is interested in the theoretical side of online advertising, check out MIT’s “Operations Research and the Internet” conference May 29 –30 at the Tang Center at MIT, Cambridge, Massachusetts.

Schedule still coming together, but speakers include Meredith Goldsmith of Google, Prof. David Williamson of Cornell, Prof. Ramesh Johari of Stanford, Prof. Balaji Prabhakar of Stanford, and Prof. Devavrat Shah of MIT.

I’m honored to be invited back to speak on “The Business And Mathematics of Pay-Per-Click Bidding.”

If you’re in Boston on May 30th and enjoy equations, do come on by.

Here’s a video from Steve speaking at Google in November (he was at Yahoo then, hadn’t yet joined Google). Audio is poor, but content is amazing. Buy your engineers pizza and have them watch this over lunch tomorrow.

The performance golden rule: 80% to 90% of the time the user spends waiting is spent after HTML reaches the browser. — Steve Souder

OpenSocial does the same for social networks.

Sitemaps does the same for web page listings.

See a pattern?

As engineers, we were intrigued. We attempted to crack the security on Google’s press release server. We succeeded. This link /cgi/private/crack.rb shows press releases coming out of 1600 Amphitheatre Parkway for the next few weeks. If there are multiple in queue, you can page them using the red “Next Press Release” button.

Bottom line: the future is about establishing standards.

These upcoming announcements show Google gets it.

Please keep this quiet so Google doesn’t patch the security hole.

Link:/cgi/private/crack.rb

(post 4/1/08)