By Joseph Raczynski

Recently some select firms have forgone the necessity to prove return on investment (ROI), for KM, portal and Enterprise 2.0.  However for the vast majority, ROI justification remains a constant.  In this session at ILTA, the panel examined the basics of metrics, how to measure productivity rather than busy-ness, how to measure engagement, and concrete ways to measure portal and Enterprise 2.0 applications.

As a baseline understanding for this discussion, metrics are numbers to gage progress, i.e. a quantifiable means to measure if there is a move from one point to another.  Firms engage in this activity to evaluate success and decide what to fund.  Simply stated they create these objectives to see if they are materially advancing the goals of the organization.

Metrics as a lens, key points to ponder:

• Focus on the factors that have the most positive correlation for success
• Ask yourself, “If success is… X, then I need to measure… Y.”
• What are the business pain points?
• Knowledge management is not always just technological issue, often it is education
• What does success look like? Is it the bottom line for a firm?
• KM tools are a critical component to success, choose wisely

Testing those metrics is a major component to your KM success.  Start with an educated guess based on your project goals and gather some basic data.  Also key to this empirical testing is to consult with focus groups and friendly stakeholders.  This not only verifies that you are working in the right direction; thus validating your work, it also tests your view verses others, establishing a complete balanced vision for the firm.  Lastly based on any feedback from these groups in testing these components, a firm should revise their methodology if necessary, adjust and repeat this process.

Portal metrics:

• How many hits does the portal get? This will give the firm a glimpse into the level of interest and usefulness of the portal.
• What kind of frequency do users return to the portal and for how long do they stay? These numbers will also assist in understanding interest, usefulness or possibly… forgetfulness.
• Your firm should gather data on the identity of users. From these metrics you can figure out patterns of adoption; that is what age group, which practice group and what geographic area is using the portal?
• Pull data on how current is the content is and to whom it is available. You can gather information on if people are using the information via a “like” button or if they are sharing it.
• View the impact on other systems or tools. Has traffic to the other applications been reduced?  Do people refer to this tool or others?

Enterprise 2.0, aka Activity Streams or Social Media metrics:

• Listen to the number of requests for access to this new tool. If you see something like Yammer jump in requests, you will be able to consider that as a major driver for implementation.
• Once the tool is in place, view the usage over time. Often these new media applications are classified as a “five day wonder”, i.e. is this tool sticky, and will the usage change over time?
• Another metric to consider is the pace of conversation for the tool. Is the tool a “go to” resource or peripheral to the organization?
• Other areas to consider viewing metrics on are the numbers of links that people add to their work product, “retweets” and the time of day for usage
• Lastly it is important to understand the impact on other tools. Is this new application used in preference of other tools, like email?

ROI is an impactful and important aspect of any business plan at law firms.  Examining the factors that have the most positive correlation for success is essential.  Once you have established a set of metrics which have been vetted by all major stakeholders, continue to test and understand that the mythology may need to be adjusted over time.  Those quantitative metrics will help guide the firm to success.