SIGMM Records

March 2009 Freely available source code, traces and test content

Freely available source code, traces and test content



ContextMiner is a framework to collect, analyze, and present contextual information along with the data. It is based on an idea that while describing or archiving an object, contextual information helps to make sense of that object or to preserve it better.

ContextMiner helps one (1) run automated crawls on various sources, and collect data as well as contextual information, (2) analyze and add value to the collection, and (3) monitor digital objects of interest over a period of time.

Currently, ContextMiner supports YouTube, blogs, and Twitter for running automated crawls. New sources will continue to be added.



TubeKit is a toolkit for creating YouTube crawlers. It allows one to build one's own crawler that can crawl YouTube based on a set of seed queries and collect up to 17 different attributes for a video. TubeKit assists in all the phases of this process starting database creation to finally giving access to the collected data with browsing and searching interfaces. In addition to creating query-based crawlers, TubeKit also has a number of tools that allow one to extract YouTube video links from a webpage, download flash videos, and collect user profiles from YouTube without running any queries.

Previous Section Table of Contents Next Section