Worksheet

Tablets

2011-03-13T09:17:00.001-07:00

I have the original iPad but did not feel the compelling need to jump to the iPad 2. It's been months since I used my iPad, not because it's not good, but because it just doesn't suit any of my current use cases. Let's review those:

Internet hosting: A server with pair networks
Heavy computing: A Mac Pro workstation, with an NEC wide gamut display, and external RAID by macgurus.
On-the-go computing: A macbook air 11 inch.
e-reading: A kindle wifi only
Mobile on-the-go: Droid x

Basically, the iPad would fit the macbook air's current spot. The problem is that the iPad has no keyboard for easy editing, and my attempts to marry it with a bluetooth keyboard ultimately just resulted in a hosed keyboard, no easy way to carry the two together.

I think the dream many report would be to marry the bottom 3 devices in a tablet. In my mind, it's an open question as to whether that can happen or is even desirable. The wifi kindle is cheap and light, easy for extended reading. Mobile phone prices and data plans are clearly headed toward commodification (though carriers are fighting tooth and nail). The replacement cycles are different.

By marrying e-readers, phones, and on-the-go computing, you make yourself ripe for getting held hostage with higher prices to follow. Just witness Apple's recent decision to force vendors to pay a 30% fee for all in-app purchases, some of which are mandated.

Posting to blogger from my Droid x

2011-02-05T04:40:00.001-08:00

This works reasonably well. I'd like to set my location as something other than an exact address. I'd also like a list of labels I've already used. What happened to all of these instant features google talks about constantly?

Software Development Priorities

2011-01-28T09:41:00.001-08:00

So, I occasionally write software to do useful things for students and analyze interactions in the classroom. One reason we're using Google Buzz these days is that it has excellent API support.

As an only sometimes software developer, I have to think about where it makes sense for me to spend my efforts. Here are my criteria:

Is it likely to be developed by someone else who will have more time to maintain it?
Does it require a lot of infrastructure to pull off?
Should it really be part of the platform but the platform provider just hasn't realized it yet?
How much time am I wasting on mindless repetitive tasks to achieve what I want?
How substantially would it help students improve their performance?
If I devoted a personal man week could I make substantial progress?

The answer to the first three questions has to be "no", or really close to "no". the answer to the last three has to be "pretty substantial" or "yes".

Getting people to contribute information

2011-01-20T06:49:00.001-08:00

My giant complaint about twitter is that it's mainly phatic. All of this talk of retweets, etc. is noise making. It gets me to look. It tells me who considers who important. Those are things to know, but they get me only so far.

I use social media a lot in my classroom teaching. I'm looking for people to contribute information which we then craft into knowledge. Some of that is phatic. Mostly though, it needs to be about substantive things students can do to effect tangible results.

Substance and results matter. If you can't produce those, your social connections won't matter.

My new cloud computer

2011-01-15T19:44:00.001-08:00

I have the 11" macbook air. It came Friday. It's expensive. It's something I said I would never get. I like it a lot.

Why'd I get it?

I need a light computer to carry around.
Its solid state drives are nearly indestructible
It offers a lot of ways to connect to cloud services, not just a web browser
It maintains the apple/unix compatibility I've been accustomed to for the last decade

We'll see how it plays out.

Quick Review of Reeder for the Mac

2011-01-09T09:03:00.001-08:00

Short version: it's fantastic. If you're on a Mac, get it now! Reeder for the Mac is available here. It offers an unparalleled Google Reader experience. It's what feed reading was meant to be. My only complaint is that it does not allow tagging posts, a critical part of my workflow. I use shared tags to organize my reader output for different constituencies.

Devices and platforms

2011-01-09T07:50:00.001-08:00

I would like devices that plugged into platforms and just worked.

All of the talk of software as a service or infrastructure as a utility is just hogwash unless I can do that. The truth of the matter is that, at many levels, software is not sufficiently standardized to be offered as a service. Further, infrastructure is optimized as a stack around a presumed application model. There is currently a war going on around what will be the application model of choice that extends all the way down to devices.

You cannot just plug devices into networks anymore. This is a step backward.

My own personal cut at the Monty Hall problem

2011-01-02T09:55:00.001-08:00

Yesterday, I bookmarked a Bayesian solution to the Monty Hall problem. The main drawback to that solution is that it requires you to go through some rather complex mechanics and recastings to get how it works. I think there's a simpler, plainly intuitive solution to be had.

But first, in a nutshell, the Monty Hall problem is this:

You're given a chance to choose one out of three doors, behind one of which is a prize.
After you make your choice, Monty reveals that one of the two doors you did not choose is incorrect.
Monty gives you the option of sticking with your original choice or switching to the remaining unknown door that you did not originally choose.

The key intuition to getting the correct solution is this:

Your original choice was one out of three. Therefore, you had a two out of three chance of getting it wrong. Nothing that happens later changes this fact. Your original choice of door continues to have a two out of three chance of having been incorrect throughout the duration of the problem.

Now, when Monty tells you one of the doors you did not choose is incorrect, the door you originally chose still has a two out of three chance of having been incorrect. Given that there is now only one other door remaining, it must have a two out of three chance of being correct. You should switch when Monty gives you the chance.

I must confess that this is not how I would have originally tried to solve the problem. Like most people, I originally assumed that I was going from a one out of three to a one out of two equiprobable decision problem, meaning that both remaining doors had an equal chance of being correct. In other words, there was no reason to switch. The better intuition is to think that Monty has effectively combined the two doors I did not choose into one.

Recasting the problem into the following equivalent form makes this clear:

You may stick with the single door you originally chose or take both of the remaining two doors you did not.

Google Buzz and Canonical Representations

2010-12-24T09:46:00.001-08:00

So, salmon sounds interesting, but one has to ask himself why, particularly if it requires waiting.

I want a Google Buzz that can do the following:

Allow me to disseminate digital artifacts to people who choose to receive them.
Allow me and others to interact around those artifacts and any others that may appear in our streams.
Allow the interaction around an artifact to be represented back to the point of origin of the artifact. For instance, Buzz interaction around a blogger blog post should be able to be shown back on that post.
Allow interaction at the point of origin of the artifact itself to be shown in the Buzz stream for that artifact. For instance, comments on a picasa picture should show up in the buzz posts about that picture.
Allows all of the functionality just described to be implemented immediately on Google properties.
Eventually allows all of this to work with other platforms, including self-hosted.
Drops this nonsensical happy talk about the semantics of activity streams.

Currently buzz does 1 and 2. There is no going back to the point of origin or "canonical representation" to harken back to the title of this post. I want it all today; no more "it will be great some day."

Rick Klau is slowly rekindling my love for blogger

2010-12-22T08:24:00.001-08:00

Rick is actually not even aware that he's been influencing me. Rather my newly rekindled love for blogger has come from following Rick's Google Reader shares and contemplating the challenges I face. Here are a two of the salient ones from this past month:

Building Websites with Blogger: This is an ebook. It's not that I can't figure blogger out on my own. Rather, it's just nice to be able to reference someone else's coherent strategy and compare my own with it. I almost always learn something new this way. The book is only $9.95 (approximate cost of the Facebook Effect but of infinitely more practical import in my current circumstance).
Insert Youtube Videos: Finally, easy sharing between Google infrastructure behemoths. Make no doubt about it, Google's key strategic advantage comes from infrastructure. Hosting videos on Youtube is like getting a CDN for free, similarly for Picasa web albums. Where Google has lagged is in making it easy for you to just link stuff together.

And two non-Rick items which feel like they've come from him anyway:

Mobile templates for blogger: Finally. Google clearly gets mobile use cases. Blogger has been, to put it politely, a bit behind the times.
Blogger's somewhat unbelievable 100% uptime for the last two months: (courtesy of Louis Gray). I'm convinced that almost no one should self-host their blog. That said, it comes down to hosting platform. Uptime, speed, and flexibility are the essential ingredients here.

The era of network provider gadgets, the new railroad trusts

2010-12-16T08:16:00.001-08:00

I'm highly networked. I have cell data plans for 3 out of 4 direct family members. We have a cable modem with somewhere on the order of 10 consumption devices on 2 subnets connected to it (this is no joke and may be an underestimate).

My thirst for new network gadgets is slackening. It's not that I don't find them useful. Rather, I don't like the fact that they're tied to particular providers.

iPad 3G: ATT
Kindle 3G: Sprint
Cr-48: Verizon
Android Phones: the cell provider who sells them to you

Isn't this what brought about the break up of the old ATT monopoly? Why is my equipment tied to my network provider? Why is my access to resources then also shaped by my provider often retroactively after I've already bought in?

Just so you know, my bottom line: I'd be willing to put up with traffic shaping if my device allowed me to hop around between providers.

The chrome web store

2010-12-08T09:03:00.001-08:00

Anyone who has given the chrome web store some consideration will probably find these speculations obvious. Nonetheless:

It's clear the chrome web store is an attempt to inject social in an organized way into the world of web apps.
Chrome web store shows that it really pays to centralize social. The real question is which parts of social need to be centralized. Google's answer is that the web artifacts in question need to be findable and "rate-able" in one location.
A question in my mind: To what extent does the chrome web store represent editorial control by Google? I think they might assert that they are merely providing a framework for aggregating opinion.
It's possible to argue that Google's whole business to date has been built around implicit polling of web page relevance. Given what they're doing in the chrome web store, it seems they think social must be more explicit.

My ongoing app engine dilemma

2010-12-03T15:32:00.001-08:00

So, I'm not a professional developer or software engineer. I've never made my living entirely by writing code, though writing code has often been critical to many of my deliverables. The key difference for me and why I don't call myself a developer has been that it's never been entirely about the code for me, it's always been about the value the code can deliver.

Developers care more about the code as a deliverable in and of itself.

Now, with that preamble, I can properly state the dilemma I face with Google's app engine. For those unfamiliar with app engine, think of it as a way to write glue code for services in the cloud. For instance, say you want to create a service where multiple authors can contribute tweets to a twitter account but you want one person to have editorial approval. You can build that app in app engine (and also, apparently, in google apps script which might be thought of as a kissing cousin).

Google handles hosting your app and making sure it has the necessary server resources.

The way I've stated it, that sound pretty cool. You can essentially write web apps without having to manage web infrastructure.

Well, the problem comes in the constraints. App engine apps can only run for a limited period. Want to do serious number crunching and analysis? That's not really what app engine is meant for, and the time constraints it places on programs likely won't permit it. You have to think about some other way to do your analysis than running it on app engine.

Therein lies the rub for the non-developer. It's relatively easy to think of how to solve these problems in a case where one program does it all. Indeed, that's how I do all my cloud computing now. I run it all on a mac mini computer with a cron job that fires off every 10 minutes, harvests cloud resources, analyzes them, and then posts the result to a server.

It's harder to think of how to split it all up among multiple resources. I can do it conceptually. It's at implementation time that I get stuck. Too many little threads to pull together, and I retreat to the way I know how to make it work.

A few thoughts on ebooks circa December 2010

2010-11-30T21:26:00.001-08:00

This post is inspired by Tracy Crawford's recent buzz about the imminent arrival of Google books. Her husband is one of the high level people at Google involved in that product. My own interest is mainly as a consumer. As an academic, I can buy thousands of dollars worth of books a year. One year, about a decade ago, I totaled $2000 on technical books. Fortunately, I had a grant to pay for it.

Lately, I've been getting into ebooks. It's a way to carry your library everywhere, and they're often dramatically cheaper. I've migrated from the iPad and iBooks to aldiko and kindle on my Android phone. Recently, I bought the kindle to improve the reading experience. It works well with both drm'd and non-drm'd content even though it's restricted to mobi and pdf.

The sum of my experience is that the kindle hardware offers by far the best reading experience of all of these platforms. Couple that with kindle's software support on multiple platforms and Amazon has packaged up an unparalleled electronic reading experience that requires very little effort on the user's part.

Here are the parts of that experience that particularly impress me:

Having access to your library anywhere is wonderful.
Having multiple device support for your library is wonderful, particularly if you can get advanced features like color on devices that support it.
Being able to sync your reading location in specific books across devices is great.

Right now, the kindle only offers full support for these features for books you buy from Amazon. This doesn't seem completely unreasonable to me. Yes, there's lock-in, but you can also make the case that book revenues are needed to support the infrastructure. Further, for how long is there lock-in really? For instance, I read pleasure books and basically throw them away. References for specific products are really only worth keeping for a year or so until the products become out of date.

There are a few technical and theoretical references that I've kept for decades. But, those are the exception, not the rule. For the general throwaway reading I do, DRM and proprietary platforms (as long as they're cheap and ubiquitous) are not the issue.

Bootstrapping a Google Buzz Learning Community

2010-09-17T18:19:00.001-07:00

The semester is fully underway. I have students loaded onto Google Buzz. I think it's worthwhile to outline my initial bootstrapped strategy for making this all work.

I'm essentially combining Google Buzz, a social technology platform, and a group of classes with the idea of creating some sort shared social space. The main benefits I see are:

Creation of a cross-class dialog. This happens in two ways:
- People follow and converse with each other on buzz itself.
- In each class, I highlight what I consider to be particularly useful contributions from all of the combined classes.
Increased knowledge generation by students. In classrooms that don't use social media, the professor is typically responsible for over 90% of the talk time. In an online social media space with mandated student participation, that percentage is effectively reversed.
Early warning about issues that students may be facing. In the past, there's proven to be a strong correlation between social media participation and performance in the rest of the class. Additionally, participating students raise issues which are often shared by many others.

Why Google Buzz?

Comparing Buzz to the alternatives, it has the following cluster of features which lend itself to my purposes:

Updates are shared publicly without the need to log in to see them. This feature vastly facilitates sharing and dissemination. Of the two major social networks, Facebook and twitter, only twitter shares this feature.
Updates can be of arbitrary length. Neither Facebook nor Twitter share this feature.
Comments attach directly to updates facilitating effective conversations. Only Facebook shares this feature.
The API is very complete and facilitates real-time updates. Both Facebook and Twitter appear at parity with this feature.

Where Buzz Needs Scaffolding

However, Buzz is missing two features that are pre-requisites for my purposes:

There is not an easy way to track participants' output so you don't miss important updates from individuals. To solve that issue, I used Google Reader to aggregate the output of all participants in one easy to track web page. Users of Google Reader, can subscribe to the aggregate bundle of participants here.
There is not an easy way to know who is in the group or see how the group members are connecting with each other. I've created a web page that solves this issue. It contains a list of class members along with information about their asymmetric and symmetric ties (i.e., following, follower and friend relationships). The page is updated every ten minutes (though you have to manually refresh the page to see the updates. The code to produce the page is contained in an open source project located here.

Higher Ed in a World of Declining Demand

2010-09-10T18:54:00.001-07:00

Demand for higher ed is actually not declining, at least overall. In fact, most headlines would indicate the exact opposite: higher ed is booming. But if you look at growth rates in enrollments, you'll note that it is high at top tier traditional institutions and very high at for-profit institutions. Enrollment is often stagnant or dropping at traditional lower tier universities.

Lower tier universities account for over 98% of the institutions in the traditional sector.

Under the current conditions, lower tier traditional universities are doomed:

They have the bureaucratic inefficiency of higher tier institutions without the resources to compensate.
They adhere to performance metrics related to academic excellence which are not directly tied to their revenue drivers.
The only institutions that really figure in the public discourse are the top tier ones.

As it turns out, their most direct competitors, for-profit universities turn these attributes on their head:

They have dramatically reduced administrative overhead and extremely lean cost structures.
Their performance metrics are tied to revenue drivers, namely enrollments.
They figure prominently in the public discourse but often very negatively. The main knock is that these institutions focus overly on obtaining up front payment for their services without delivering results such as improved job prospects.

In light of these data, the only real hope for lower tier universities is to compete on demonstrated student outcomes by adapting their programs to current market conditions. But they often can't or don't due to bureaucratic inefficiency and the fact that they do not adhere to performance metrics that drive revenue.

A buzz network crawler

2010-09-06T20:28:00.001-07:00

Last but not least in the last few hours before summer unofficially ends, I've updated my buzz network crawler that I first released back at the end of May. The crawler is code written in Python that follows symmetric ties on buzz networks to an arbitrary depth. It makes use of Google's Buzz REST API.

The new code is a major update with the following features:

The user may specify the initial seed network of buzz users. Typically, this seed network will be some group of people who are socially connected. In my case, it's always been students in a class, but it can be anyone. The members of the seed network may be specified with email or numeric IDs. Both are shown in the examples.
The user may specify an arbitrary crawl depth beyond the initial seed. For instance, by specifying a craw depth of 2, the crawler will crawl the initial seed group, their connections, and the connections of those connections. It should be noted that crawl depth appears to increase run time exponentially. The crawl to depth 2 for results described here and here took approximately 22 hours and encompassed 10,000 network members.
The crawler uses a memoization pattern to reduce expensive calls to the buzz API for network members with multiple inbound connections from other network members (applies to almost all members).
The crawler accounts for and gracefully handles known deficiencies in the Buzz REST JSON API. Specifically,
- The crawler is able to process non-standard control characters embedded in JSON return documents.
- The crawler recovers gracefully from 503 Errors occasionally returned by the API when the user is following a large number of people.

Known Limitations

The code should be considered alpha quality. As I was writing it, I had to simultaneously:

figure out the problem(s) I was actually solving. This should be apparent for anyone comparing the capabilities of the initial release with this one.
Learn new programming constructs in the Python language. Not least of these was discovering an effective mechanism for managing key-object databases.
Develop an effective strategy for reducing run time to manageable levels.
Develop a strategy that was robust against API errors. This was critical as crawl times of 24 hours or greater are sometimes required.

Documentation is minimal and contained in the README files found in the code download.

This post would be remiss without recognition to the many Google employees who gave me help in the Buzz Google Group.

Thoughts on the Summer of Android

2010-08-29T12:37:00.001-07:00

For me, the summer of Android, when I really discovered Google's mobile operating system, started at Google I/O:

Froyo, the new Android version was launched.
We were given the additional conference gift of the Sprint EVO 4G, then arguably the most advanced Android phone available.

Like Saul on the road from Tarsus to Damascus, my eyes were opened within a day of possessing the EVO. Since, I've purchased the Droid x for myself (an upgrade from my original Droid) and have been following all telephony, not just mobile, closely. The summer has seen a tidal wave of telephony developments from Google and its partners.

Here are my summary observations about where we stand today, the weekend before summer informally ends:

The ability to make calls from Gmail when combined with Google Voice is simply game changing. Suddenly, I have the best, free telephone routing system (Google Voice) liberated from the need for a physical device. At this juncture, I don't need separate POTS (plain old telephone service), and I only really need mobile for when I'm out of contact with a land-based network.
It's easy to see the day when everyone in the phone business will just be a data network provider. Coverage and speed for price will be the defining characteristics.
Since I now view mobile connectivity as a supplement to and not a substitute for land connectivity, I'm most concerned with net neutrality for land-based networks and trunk lines. That's where most connectivity is playing out for me now. It turns out mobile is a specific use case where I want to access specific things. In other words, I can see how prioritization might make sense in the mobile scenario.
The biggest value in my mobile phone is my ability to connect back to "web-based" services that add context to my current situation, for instance, all Google location-aware services.
The next biggest value in my mobile phone is the ability to connect to my "social graph" (i.e., the network people I'm connected to via some electronic service) in all of its forms. Android-facebook integration is useful in this regard, but the real ace in the hole is the contacts manager, and I think eventually, profiles. By the way, since I've had to resort to various Google Reader hacks to follow aspects of my social graph, I'll throw that in there too.
The third biggest value in my phone is my ability to use it to access reference information in all forms, be it electronic books or the web.

Note how making voice phone calls is not listed as a separate item. I do this infrequently and would tend to include it as part of connecting with my social graph. I should point out that this is a giant change from where I was at the time of the ATT breakup in 1984. Then, I was excited by the new plethora of voice-related communication options. Simply put, times have changed. Text has replaced voice in most applications.

Google Buzz for Engaged Academic Publishing

2010-08-24T14:31:00.001-07:00

Even if you're not in academia, if you read this article from the New York Times, you'll realize that academic publishing is undergoing upheaval. Essentially, it's the same upheaval that's upending publishing in general: The quality of what's disseminated would improve dramatically if it were subject to continuous revision based on feedback from as large a group of people as possible who have information to bring to bear. It's just that the established players are having a hard time understanding, let alone adopting, the new quality model.

The longstanding quality model to be replaced in academic publishing is prior peer review. In prior peer review, articles are not published until they pass muster with a small group of experts, usually two to three people. The notion is that these experts will carefully consider the evidence and conclusions presented in any given work, only publishing those works which pass the quality test.

In the remainder of this essay, I'll first outline why prior peer review is the problem not the solution for the dissemination of quality academic research. Then, I'll present one approach I've cobbled together with Google Buzz.

Prior peer review is the problem, not the solution

The problem with prior peer review is that it effectively limits the scope of review to just a few people, some of whom may not be particularly qualified or may be operating on an agenda that precludes publishing certain, otherwise meritorious pieces. Even in prior peer review's most benign form, it implies that work must fit within the current consensus, which tends to change infrequently at best.

I once had an article go through peer review at four separate outlets over the course of a decade before getting accepted for publication.

When I first started trying to get the article published, it was a bit out there. It applied cutting edge statistical techniques to understand the effectiveness of emotion-laden persuasion tactics used by credit collectors. The article is now available for purchase online from a prestigious journal at the single article price of $30. If you're really interested, I'll be happy to send it to you gratis. I didn't do it for the money.

For all ten years this article was under review, its fundamental conclusions did not change, though there were additional analyses performed and re-packagings undertaken. The end result was a piece of writing hidden behind a paywall. In other words, ten years of effort for non-dissemination beyond the journal's subscribers.

Enter Google Buzz

Given this tale, one can reasonably wonder what Google Buzz has to offer. After all, it doesn't seem in any way to match the process I've just described.

When trying to publish your work, there are three things that matter:

Reach
Targeting
Feedback

In the prior peer review model, the feedback that matters comes from the reviewers prior to any dissemination. Targeting is based on what your colleague's read. Reach is typically limited to those colleagues and their cohort.

Now, consider Google Buzz. For significant efforts, I typically get feedback from 10 or more people. Often a number of people will reshare the post, sending it on to their audiences which often number in the thousands. Some of these individuals alone have greater reach than any, other than the best-known, academic outlets.

But, do any of these Google Buzz people know anything?

To use Buzz effectively, you need to target people who know something. I've basically cultivated Buzz associations based on my intellectual interests. It turns out that these intellectual interests all relate to the extremely applied research agenda I am pursuing in Internet APIs, online social networking, education, and (oddly enough) web marketing. I'm getting feedback from people highly knowledgeable in all of these areas. I suspect this feedback will particularly help me on the software end of the endeavor as well as in understanding some of the implications for online social networking

But is it academic publication?

One thing my Buzz postings clearly are not is dissemination to a small group of experts after approval by an even smaller group of experts. In that sense, they are not academic publication.

However, it's important to remember that academic publication started out as the exchange of letters between people who thought they could profit from talking to each other about the work they were engaged in. I'm essentially using buzz as the envelope for a similar endeavor.

I would describe the buzz posts I've worked on and crafted for feedback as having achieved the kind of engagement I wanted: specifically feedback from highly knowledgeable people who, in some cases, have built significant businesses using the tools and social networking phenomena I'm investigating. Two recent examples of posts that illustrate this dynamic are: the distribution of friend connections among Buzz users relative to Facebook users, and how to infer a person's online social connections even if they keep them private.

So, my personal answer is that Google Buzz is providing me a platform for more engaged academic publishing than what I achieved through the more traditional route.

What's the long term prognosis?

To a large extent, the importance of what you do depends on how it impacts others. I'm doing everything I can to have impact, and enlisting knowledgeable input from others strikes me as important in achieving that.

I'm somewhat less concerned by the sociology of academia. Demonstrating impact tends to trump all.

Laporte's lemma: A workaround for Buzz lists using Google Reader

2010-08-23T15:40:00.001-07:00

I need to thank Gord Wait for putting this idea back in my head in an actionable way. If you're a buzz user, you may feel beset by noise. You're following a lot of interesting people, but some of them post infrequently such that their posts are hard to discern in the flow.

There have been many calls for twitter-like buzz lists to solve this problem. Essentially, you would create lists of people whose updates you wanted to track, and then by clicking on the lists, you would see only the updates from those people.

Such lists have not been forthcoming from the buzz team. However, as Gord reminded me, you can simulate such a list by using the atom representation of the person's public activity stream, and then grouping all of the people you want to follow into a folder in reader.

The question is: What is the url to that atom stream representation? Well, it turns out it has a canonical form that looks like so:

https://www.googleapis.com/buzz/v1/activities/UserNameOrNumericID/@public

For instance, if you wanted to follow me this way, the following URL with my user name would suffice:

https://www.googleapis.com/buzz/v1/activities/fpgibson/@public

Some people don't want to reveal their user name, so you're only able to get their numeric ID. It's the exact same format. Here's me via numeric ID:

https://www.googleapis.com/buzz/v1/activities/114242352345417873286/@public

In each of these URLs, the part in green represents the service you are calling, and it never changes. The part in red is a modifier indicating that you only want what the poster chooses to share with everybody. That also never changes (You can't track private posts in Google Reader because it does not support the authentication required).

This is how I'll track the 70 students I'll have using buzz in two weeks.

So, why is this called Laporte's lemma?

The title is meant as a humorous take on Leo Laporte's abandonment of buzz this past weekend because he felt people didn't notice his stuff anyhow and Louis Gray's response that that might be because he, Louis, had so much other stuff in his stream. As a result, Leo's absence went unnoticed by Louis.

In the world of mathematics, a lemma is a little side proof one does in getting to the main event or it is an easy to derive consequence of a proof that has practical import. It's in this latter sense that I meant the word in the title.

Inferring Your Ties on Google Buzz Using What Your Friends Say

2010-08-22T13:51:00.001-07:00

A key component of privacy in social networks is the extent to which your connections and associations are public. For instance, Facebook makes your friends' names and IDs available to any application you use, though it allows you some control over how this information is revealed on your profile page. Google Buzz allows you to remove the lists of people you are following and who follow you from your public profile. An interesting question for a Buzz user is how effectively this feature allows you to hide your ties to others from public view.

The graph above shows that in a sample of over seven thousand Google Buzz users, I was able to infer approximately 40% of their ties without needing to refer to their reported ties. I just used the ties their friends reported. With a sufficiently comprehensive crawl, this percentage would approach seventy, or the percentage of people I estimate to make their following and follower lists public.

In other words, in social networks, what your friends say about your ties reveals a lot, even if you yourself keep the information hidden.

How I performed this analysis

Using the student-derived data set I reported on previously, I did the following:

I collected the follower and following information of 7,225 network participants reporting their following and follower lists publicly.
I counted a tie when one participant appeared in both the following and follower lists of another participant. This method allowed me to infer when a person who kept their lists hidden was tied to another person.
For users reporting their ties publicly, I plotted the relationship between inferred and reported ties.
Regressing reported on inferred ties for these public reporters revealed that for every inferred tie, the person reported approximately 2.5 ties. Stated otherwise, inferred ties represent 40% of reported ties (n.b., 0.4 or 40% is the inverse of 2.5).

That's not the end of it

One might assume that if everyone kept their following and follower lists hidden that that would be the end of it. Well, not really. Ties can simply be inferred based on public communication patterns. The lesson here is that the extent to which any of your interactions take place in a public space, inhabitants of that space will be able to infer things about you and the people you are connected to.

Areas that require further work

As in my prior post, my sampling approach here is not random. In particular, my students were following people who they could find publicly, so the estimate of the percentage of people hiding their following and follower lists is likely low. Further, I'm assuming that people who hide their following and follower lists are similar to those who report them publicly.

The solution to both these issues is better study design with random sampling. Further, the issue of hidden follower and following lists can be addressed by getting those users' permission to access their lists.

Do buzz users have more ties than Facebook users? 10,000 answer

2010-08-12T14:26:00.001-07:00

The graph above is based on an analysis of 10,113 buzz users where 7,225 of them chose to share their "following" and "follwers" lists publicly. In the graph, I'm counting Buzz user ties as occurring when two users each follow the other. In other networks, like Facebook, ties are equivalent to friend relationships between users.

The distribution of ties is quite clearly skewed to the left:

The maximum number of ties recorded for any one user is 4,754.
However, the vast bulk (90%) of users have 394 ties or less.
76% of users have the mean number of ties (159) or less.
The median user (the user at the half way point in the list) has 50 ties.
In other words, half the users on buzz have 50 ties or less.

So, do Buzz users have more ties than Facebook users?

As of today (8/12/2010), the average Facebook user has 130 friends. If we look at the mean number of ties for a buzz user in this sample, 159, then buzz has a slight edge but is clearly in the same ballpark.

In the case of either buzz or Facebook, we have to recognize that the ties are nominal. The level of ties actually maintained on either network is likely just a fraction of the nominal number as suggested by an in-house analysis done by Facebook.

How I collected the buzz data (why you should take this with a grain of salt)

The data is based on a snowball sample originating with myself and my 10 web marketing practicum students. I then went to the friends and finally to the friends-of-friends levels. The data were collected from August 4 to August 5, 2010. Here is a summary of the number of users analyzed at each level:

Core group (myself + students): 11
Friends: 161
Friends of friends: 9,941

A few points about this sample:

The students all have fewer than the median number of ties. While I have over 100 or double the median publicly reported.
In other words, all of the outliers with many ties come from the Friends and Friends of friends levels.

This latter illustrates the problem with snowball sampling. It's hard to know what aspect of it is representative of the population as a whole. A random sample would alleviate this problem.

Going forward

I began down this path in an attempt to keep track of the networks my students were forming as we proceeded through the semester. The area that interests me most is the quality of ties they are forming. Going forward, I'm most likely to push on content analysis and network structure.

Future Posts

The mathematically inclined will have already noticed that 29% of my sample did not want to share their following and followers lists publicly. I'll be providing more analysis on this group in a future post.

Tracking the evolution of symmetric ties on Google Buzz

2010-07-05T07:11:00.001-07:00

At the end of May, I introduced a tool for monitoring the up-to-the-minute status of my students' bootstrapping efforts on buzz (the code is available as open source software here). Today, I'm introducing another tool for tracking the evolution of symmetric ties in Buzz networks. Symmetric ties, where two people mutually follow each other's updates on the network, are important because the more symmetric ties a participant has, the more possibilities they will have for conversation and hence the more useful they will find the network.

You can see an excerpt of the tool's output in the image above. It's basically a web page where participants in the Web Marketing Practicum see three columns beside their mini profiles:

The number of symmetric (mutual) ties they had as of the analysis date, along with links to all of those people's profiles.
The number of symmetric ties that they added since a reference date, again with links to the actual profiles.
The number of symmetric ties that they subtracted since a reference date, again with profile links.

Using this information, participants can see who is entering and leaving each other's mutual ties networks. Perhaps more importantly, discussion can ensue about the type of network each participant is trying to grow and whether the additions and subtractions that have occurred make sense in that context.

A few quick remarks:

With one exception, every student has at least doubled the size of their symmetric ties network since May 31, 2010 when I did my first analysis.
In the two weeks covered by the pictured analysis, students typically increased the size of their symmetric ties network by over 20%.

How the tool might be useful

I was interested to read Adewale Oshineye's Buzz post where he, in effect, wondered why people would be concerned with symmetric ties on Buzz. So, it seems useful to list a few reasons why a symmetric ties tracking tool might be useful:

It's hard to imagine people exchanging information on a sustained basis unless they are mutually following each other (i.e., have symmetric ties). Knowing that your symmetric ties network is growing is an indicator that the network is a useful source of exchange.
People, being people, tend to seek reciprocation in their social relationships. They find non-reciprocating (asymmetric) ties less appealing and will deemphasize networks where such ties abound. Knowing who is in your symmetric ties network and how it is changing over time is a useful indicator of the kind of value you're getting out of it. Is the network you're growing on the platform a colleagues network, a family network, a friends network, or what?
In general, it pays to have visual indicators of effectiveness. Buzz currently lets you know how many people you are following and how many are following you. However, there is strong evidence that high follower counts do not translate into reach. On the face of it, symmetric tie counts might be a better indicator (but see limitations below).

Tool limitations

The analysis is purely descriptive. It only shows how one aspect of the networks has evolved. It doesn't shed any light on why or how that evolution occurred.
The way the tool is set up, it suggests that gaining more and more symmetric ties is a good thing. I believe that bias to be useful when you're starting out. However, at some point, it doesn't make sense to try to increase your symmetric ties beyond what you can effectively track.
The tool places no weight on the quality of the tie. Is this someone it makes sense for you to be tied to?

Future directions

If you step back, you realize that Buzz is one instantiation of the next phase of personal publishing. A number of thoughts arise:

I'm interested in knowing more about the connection structure in these networks. How are people down through the friends of friends structures connected?
I'm interested in expanding the analysis beyond the current core group of people. If I go to friends of friends, I start to have enough data to make computing individual page rank interesting.
I wonder to what it extent it would pay to make friend recommendations from this data.

Getting the code

Once I have a chance to clean the code up, it will be available on my buzz list tracker project website.

A Buzz API Python Application for Social Networks Applied to My Students' Buzz Networks

2010-05-31T18:53:00.001-07:00

Click the image to see the results of a project I've been working on for the last week. My goal was to start to get a handle on how well students in my Web Marketing Practicum class were getting on in Buzz. What I produced was a basic tabular report designed to show how well the students were connecting with each other and with external parties.

Here's how to read the columns:

Participant: Just the class participant whose network is being examined. I'm included in this as I'm part of the network.
Mutually Following: The people the participant is both following and being followed by. For many social network analysts, this column is what constitutes the social network. Other participants in the class are color coded orange and in a bold weight font. Those outside the class are blue and in a regular weight font. A quick perusal of this column reveals that, with the exception of myself, the majority of each of the other participant's networks is composed of other participants.
Not Following Participant Back: Often, this column is not important. It may represent people who the participant is following purely for information. However, it can start to be an issue of poor perception management if no one the participant follows is following them back.
Participant's Other Followers: These are people who follow the participant but are not followed by the participant. They may represent an opportunity for the participant.

I'm sure this analysis seems super simple. What's the upshot?

One issue with Buzz as it stands in May, 2010 is that it does not have an easy way to perceive your network. Who's in? Who's out? Who are potential people to connect with. This analysis begins to provide an answer to all of those questions.
Right now, I'm clearly the most connected node on this network by any measure. Students may be able to feed off of me. Also, some of them have started to grow their networks, and as they do so, they can feed off of each other.

My plan is to do a separate post on the Python code itself sometime in the next 10 days, and I'll include a cleaned up version of the code with that. Suffice it to say that I did not use the Python client libraries for the Buzz API. Rather, I just used the RESTful API. The main reason was that it was chock full of examples for how to get the data I wanted. I did wind up writing a simple Python abstraction layer for it.

Next Steps

To be honest, it may be seeing wether I can duplicate this effort with the twitter API. All reports indicate that my students may be having an easier time there. Tracking is certainly easier. I just created a list for my students.

However, even with twitter, figuring out who is in your active network is hard. As simple as this exercise is, it begins to accomplish that task.

A plan of attack on the buzz API using Python + REST

2010-05-25T18:14:00.001-07:00

So, as of last Wednesday, the Buzz API is out. Right now, things are early stage still. You can do things, but there are not a lot of refined examples to help you along, and the client libraries for popular web programming languages like Python are not yet fully complete.

By counter, examples for the REST endpoints are more fully developed, and I've decided to just use those. Directly using the REST endpoints in python is a little more difficult because it requires you to do things like marshall your requests into a non-python data format such as xml or json and decode all responses from said format. You also potentially have to handle things like authentication using the oauth protocol. Fortunately for the intrepid, there are a few resources that can shed some light on how to proceed. I'll list them here:

First is the documentation for the buzz REST API. It gives both an overview of the API's philosophy and concrete examples to illustrate it. Often, you can just execute the examples in the address bar of the chrome web browser to get an idea of the kind of information you'll get back.
Second is oacurl. This is a java command line utility that's pretty easy to install on most unix-like systems (I'm on mac os x). It's an easy way to make requests of the API and see the raw response it sends back. When required, oacurl allows you to easily execute actions that require authentication via oauth. The cookbook is invaluable. Executing those examples will give you more than one aha! moment.
Third is the buzz bingo example written in Python and presented last week at Google I/O. The beauty of this example is that it follows the REST strategy I'm laying out for myself here. In other words, it can act as a blueprint in Python for some of the features and hurdles I'll need to overcome when using REST.
The python packages httplib2 and simplejson. The main advantage is that these are in widespread use. Further, I myself have used them, so getting reacquainted should not be too bad. httplib2 facilitates connecting over http to REST endpoints. simplejson is a python package for decoding json.

My plan of attack going forward

My guess is that the developers of the buzz API expect the modal programmer using it to be coding some sort of web app which will ask the user's permission to access data. I plan to get there at some point, but first I'm going to see if I can pull public data for a few users and see if I can interrelate it. That will allow me to cut out a lot of complexity, like hosting the app on the web or needing to manage authentication.