Metrics For Open Practice Projects
Posted on October 2, 2006 | Tags:
Just a quick post with some thoughts I was having last night while setting up our first (rough) Open Practice page about metrics.
One of the tough problems for our market is how to keep track of and measure progress, success and failure. Solving this means coming up with a way to “score” our projects. Clearly, the ultimate proof is in the pudding — either the site accomplishes it’s business goals or it doesn’t — but that’s somewhat out of our hands. There also may be great value for the Open Practice in projects that don’t completely succeed in their mission.
One of the things we’ve got to break out of is this “win all the time” business mentality. What organizations need is accountability, not internally-oriented spin creating an illusion of success. There are far too many entities out there that have no way to recognize failures, to process and learn from mistakes, and also to identify and salvage the individual pieces that worked from an overall puzzle that didn’t.
We have a lot of ambitious ideas and clients, and we’re consciously putting ourselves out there as innovators. It’s part of our job to sometimes fall flat on our faces, and to make that a net-positive by taking away valuable information. That’s what experimentation and the empirical method are all about.
Metrics
I’m getting a little off track. Running this Open Practice is partly about the ethic of experimentation, but it’s also about getting data. Quantitative analysis, baby. That means metrics.
Think of it like this: if you had a baseball card for your website, what stats would be on the back? Some standard ideas:
- Number of users, posts, comments.
- Email list signups.
- File downloads, video views, audio listens.
- General site traffic / Google analytics.
Those are pretty much established best-practices by this point. I’m thinking about how to push the envelope a bit. First of all there are some more pointed user/community metrics that could be implemented:
Metrics For Community
- Ratio of successful logins to failed attempts.
- Number of user-to-user connections on a social network / median number of connections per user.
- Number of user-invites, emailed pages, etc.
- Average logins, posts, comments, comment/post ratings per day/week/month
Finally, from a the technical Drupal perspective, it might be good to have some code-level statistics to indicate the overall cleanliness of a project, how portable and scalable your success might be:
Metrics For Code-Quality, Scalability, Kludgyness
- List of contributed/custom modules in use.
- Lines of contributed/custom code, including PHP pages, blocks, etc.
- Total size in bytes of configuration: variables, views, cck, etc.
- Number of database tables, total db size.
- Avg. queries per page-view, page load time, etc.
I’m going to continue developing these ideas. It should not be too hard to work these things up for our Open Practice projects (like NAN), and I’m looking forward to producing some prototypes.
Metrics Party
I’m a huge fan of metrics. Once upon a time I proposed that the CivicSpace release and organization get heavily into the metrics game… See deets below. The development of this was never prioritized or executed. Some of this stuff is now dated, but much of it I think would be a real boon to site owners and to groups of owners seeking to measure the impact of their affiliated networks of sites… How cool it would be if ChapterThree built the now-relevant pieces of this… ;)
———
“CIVICNET” DATA NETWORK
Proposal Summary:
Development of modules that allow
CivicSpace-based site administrators to opt-in to sharing data about the use of
their site with CivicSpace the organization and/or with a network of other
CivicSpace-based sites.
Goals:
1) For CivicSpace the organization to track adoption and,
subsequent to adoption, intensity of use of the CivicSpace platform. This is critical to our ability to a)
raise money and b) monitor our impact / get an ongoing reality check about how
useful our current release of CivicSpace is.
2) For CivicSpace-based sites to be able to help each
other by sharing quantitative information about what works and what doesn’t
work in using CivicSpace to build a community.
3) To instantiate a ‘network effect’ benefit to using
4) To facilitate the creation of a ‘business
intelligence’ data asset (“CivicNet” data) with potential commercial value; a
specific application of this data asset could be, for example, a network of
partisan political sites that contribute raw data from their site and receive
in return a refined data product that helps them to operate more effectively as
a political force.
Major Work Products:
Each major work product can be developed and
rolled out independently sequentially over time.
abstracts data to create the CivicNet data package that can be requested
by CivicSpaceLabs.org server
will
up
(typically would be hourly)
less frequent pings from CS the organization
from CivicNet data
data-contributing CivicSpace-based sites
shows data from report recipient’s site(s) in context relative to leading
sites in each data category and in context relative to overall network
Query and Reporting Interface: (optional)
CivicSpaceLabs.org to run custom queries and generate custom reports off
of archived CivicNet data
service accessible to paying members of CivicSpace user community and/or
3rd party clients
CivicNet Data Package: (what we want to request from a civicspace site
server)
URL
Type Taxonomy: want to have
sites self-categorized (e.g.: political, campaign, environmental,etc.)
Geographic Scope: want to have sites self-categorized (e..g.: California,
San Mateo, USA, global, etc.)
Keywords/Metatags: want to have sites self-described with relevant
keywords
Site Activity:
/ Aggregator Module Activity:
Activity:
views
karma
word
Module Activity
Module Activity
Mailer Module Activity
recipients
Module: (future)
Modules I’m forgetting?
Flow;
Types:
and Platforms
Engines
totally on point
Metrics are SOOOOOOOOOOOO valuable. You can’t argue with cold, hard facts and numbers. Charts, graphs, trends, and more…
I likes it.
f
Post new comment