A Call to 5min of Action for More Perfect Archive

The Metavid Archive has captured video and text captions from cable broadcasts of public domain house and senate footage since 2006. We have made all this content available for search and reuse in an entirely open source video platform. But the archive is not perfect, over-the-air cable broadcasts do not provide perfect sync for close captions, and live transcription is not 100% accurate.

We are now calling on visitors to share and promote the Metavid site in order to build a more perfect archive. Specifically, we are asking people to try and spend a few minutes of their time to try out the transcript improving tools. For more help on how the transcript improving process works, check out the improving transcript accuracy help section here on the wiki.

To find a transcript to improve simply search for the issue your interested in be it Iraq, Afghanistan waterboarding, telecoms immunity, FISA, Guantanamo, same-sex marriage, immigration, or the recent bailout debate. Or pull clips up by people. If you find a transcript slightly out of sync simply adjust it, that way it will be perfectly in sync for the next person

Improving transcript segments goes a long way here on metavid because the metavid platform builds on the most powerful collaborative knowledge production platforms in existence: mediaWiki (the software that powers wikipedia), and semantic mediaWiki a powerful structured data extension to mediaWiki. Your freedom to collaborative is protected by creative commons by-sa license ensuring your freedom to reuse the archive in its entirety for any purpose as long as you don’t prevent others from doing the same.

The More Perfect Archive We are Building

These same improved transcripts are carried over when people embed posts in blogs, enhancing the content accessibility. The transcript is exportable in the open cmml timed text format it can be muxed with the ogg stream for archival distribution and is easily searched as the text is directly in the page or accessible in machine readable CMML. (not hidden or encapsulated in a proprietary player like the approach of some flash subtitle sites) Annotative layers can categorize larger stream segments of video enhancing searchability and contextualization of media segments.

Since your participation in the metavid archive semantically tags time segments and we scrape information from a half dozen open congress sites; with your participation powerful semantic queries become possible.

The site give users powerful tools to create pages that highlight particular issues for example see the bailout coverage page, and provides endless mashup opportunities. We will continue to improve the archive as we make edits to transcripts. We will continue improve the underling open source software and hopefully lay the groundwork for future collaborative video archive projects. How perfect an archive metavid becomes is only dependent on our imagination and collective participation.

Add comment dale

Native Theora for Firefox 3.1!

Looks like the code to support ogg vorbis/theora playback has made its way into the nightly builds of firefox! This is a really great development! Mozilla’s has announced support for native theora support in ff 3.1. This will have a hugely positive effect on promoting free formats and ensuring current “non-free” formats stay cheep or free-as-in-beer. This is particularly bold of Mozilla in they are pioneering a “more free web” than the standards groups were able to agree on. As previously discussed some industry participation in the htm5 group discouraged a free baseline web video format citing submarine patent concerns. The power of the premier open source projects to ensure support of a free software ecosystem can not be overstated. Mozilla role here is hugely important and its great to see they have taken the right path to ensure the possibility of a bright future for free and open media.

Update: see Ginger’s excellent summary with historical contextualization, Mozilla hacker Robert O’Callahan’s Why Ogg Matters post, coverage on bush coder blog (the firefox video integration branch developer) and Greg Maxwell’s post (one of the key supporters of ogg media on wikipedia)

6 comments dale

Open Government Meeting

Carl Malamud & Tim O’Reilly hosted an open government meeting in Sebastopol CA which was very productive. Principals of Open Government Data were established and strategies for pushing openness in the current government data environment where shared. Metavid Wiki was presented in the short demos section of the meeting.
From the announcement:

This weekend, 30 open government advocates gathered to develop a set of principles of open government data. The meeting, held in Sebastopol, California, was designed to develop a more robust understanding of why open government data is essential to democracy.

The Internet is the public space of the modern world, and through it governments now have the opportunity to better understand the needs of their citizens and citizens may participate more fully in their government. Information becomes more valuable as it is shared, less valuable as it is hoarded. Open data promotes increased civil discourse, improved public welfare, and a more efficient use of public resources.

The group is offering a set of fundamental principles for open government data. By embracing the eight principles, governments of the world can become more effective, transparent, and relevant to our lives.

Add comment dale

Openness Matters. RSS is just the beginning…

I just ran into Nicholas Reville of the Participatory Culture Foundation article on the future of internet video and how we can aim to push it toward more openness.

It is well written and questions the single service provider model that internet video is currently operating under.

Will internet video be centralized in huge services like YouTube or Google Video, or will it be more broadly distributed? (with technologies like RSS)

Some more questions to consider in the interest of “open” video on the internet:

1. Does the codec matter?

Google has fully embraced the flash video the codec. The software architecture has laid the foundation for a walled garden distribution mechanism. Video unlike images and text could be blocked from competing video search engines disabled in external players, and mediated by annoying market conditions ie advertisements. Fortunately we can still extract the flvs but flash does support locking these files away. Ogg theora on the other hand, is never locked away in a proprietary playback system, and will always be open for reuse inside our own gardens or by secondary service providers opening up the mediations of these cultural assets to everyone.

2. Beyond RSS what about arbitrary segment reference and dynamic sequence building?

In the metavid project we make arbitrary segments of video clips dynamic available for reuse in sequences on our site or off it. The meta data is freely available in its entirety, opening it up for secondary service providers and video content aggregators. For example any search in metavid can be a channel in democracy player. In time this may enable the types of rich interoperability we see happening with text in the blogospher. An environment where participants mix, match, tag remix visual content from a variety of video from a variety of sources with or without intermediary service providers which bridge the technoliteracy divide. Open systems like RSS and open API for hosting service providers enable secondary service providers like search engines to create much richer audio/video applications.
3. What would “videoPedia” look like?

Transparent versioned video Sequences pulling form huge repositories like archive.org and up to the minute video contributed by participants… collaborative editing tagging and aggregation enabling real time channels on given subjects/themes/memes. ie ~Real Open Source Television~

anyhow…I think the future is bright, even if we end up in a mostly google mediated reality its still much more open than the existing broadcast model… if we can fully enable participatory culture in the visual medium even better!

Add comment dale

Openness Matters. RSS is just the beginning…

I just ran into Nicholas Reville of the Participatory Culture Foundation article on the future of internet video and how we can aim to push it toward more openness.

It is well written and questions the single service provider model that internet video is currently operating under.

Will internet video be centralized in huge services like YouTube or Google Video, or will it be more broadly distributed? (with technologies like RSS)

Some more questions to consider in the interest of “open” video on the internet:

1. Does the codec matter?

Google has fully embraced the flash video the codec. The software architecture has laid the foundation for a walled garden distribution mechanism. Video unlike images and text could be blocked from competing video search engines disabled in external players, and mediated by annoying market conditions ie advertisements. Fortunately we can still extract the flvs but flash does support locking these files away. Ogg theora on the other hand, is never locked away in a proprietary playback system, and will always be open for reuse inside our own gardens or by secondary service providers opening up the mediations of these cultural assets to everyone.

2. Beyond RSS what about arbitrary segment reference and dynamic sequence building?

In the metavid project we make arbitrary segments of video clips dynamic available for reuse in sequences on our site or off it. The meta data is freely available in its entirety, opening it up for secondary service providers and video content aggregators. For example any search in metavid can be a channel in democracy player. In time this may enable the types of rich interoperability we see happening with text in the blogospher. An environment where participants mix, match, tag remix visual content from a variety of video from a variety of sources with or without intermediary service providers which bridge the technoliteracy divide. Open systems like RSS and open API for hosting service providers enable secondary service providers like search engines to create much richer audio/video applications.
3. What would “videoPedia” look like?

Transparent versioned video Sequences pulling form huge repositories like archive.org and up to the minute video contributed by participants… collaborative editing tagging and aggregation enabling real time channels on given subjects/themes/memes. ie ~Real Open Source Television~

anyhow…I think the future is bright, even if we end up in a mostly google mediated reality its still much more open than the existing broadcast model… if we can fully enable participatory culture in the visual medium even better!

Add comment aphid

Metavid Moving Forward ;)

We have been working on getting more content in the archive and easily accessible. We recently acquired a new 2U dual core Xeon transcoding server with the resources from the sunlight min-grant…. This new computer should make transcoding around 3x faster. This should enable quicker posting of archival content. Also we are still looking for a way to put this capture box in the capital. This will enable live and instantly archived streams of house and senate proceedings.

Work on metavid continues to support not only viewing these proccedings but enabling their reuse in digital dialog.
Development is moving forward even without a full time developer To review the new features mentioned in the metavid svn over the past two weeks:

• unified metadata table for search queries and results. (queries now run 2-3 times faster with fewer JOINS)
• people links, profile pages,
• advanced search functions in the web view
• updated playhead for ogg media (a vlc limitation makes it not work for annodex hopefully will be resolved)
• updated context playhead. (will explain more shortly)
• updated ivtv drivers for capture box
• updated archive maintenance scripts

These Features and more should be showing up in the bleeding edge section after they are fully tested and integrated by mid-late September.

Add comment dale

Metavid Thesis online

The metavid project thesis papers are now available online. Dale’s paper gives a detailed account of the metavid project its motivations and methodology.

The paper publishes details of C-SPAN’s take down request. The requests details are published to illustrate how C-SPAN blurs the line between government produced public domain material and C-SPAN produced content. C-SPAN restrictions on content are shown to limit citizen’s ability to techno-mediate these public media assets.

In this context mediation or the negotiation production of meaning is restricted to officially sanctioned mediators. Established news media transform and compile public media assets through segmentation, story selection and editorializing into consumable proprietary encapsulations.

Metavid addresses the status quo of contemporarily legislative archives by doing the following:

• Metavid returns trademark encapsulated media assets to the public domain, indexing metadata and provides a freely re-usable online access to the entirety of the archive.
• The Metavid code is open source and it’s implemented with entirely free and open source software, making it freely contestable, transparent, and improvable.
• Metavid implements the basis for a participant remediation system where mediations of public media assets are made open, transparent and contestable.

Watch for a link to aphids paper (on the same topic)

Add comment dale

Licensing Openness the aGPL and the metavid code base.

here is snipit from my thesis about the metavid code licence. The full paper will be available in a open contestble format shortly I belive aphids version will be as well

Licensing the metavid code base was an important element of the praxis of open source. Several licenses were considered to secure the freedom of participants to re-negotiate the mediations enabled by the metavid software. This survey and license selection process addressed the problems of contemporary archival systems, through the practice of open source licensing.

BSD styled licenses are generally considered the most liberal licenses and are comparable to releasing code under the public domain. The metavid code base under a BSD license would mirror the public domain qualities of the media assets that make up the metavid archive. But a BSD style license was not selected because metavid participants did not want mediations based on the metavid codebase to become closed source and lose the qualities that open source praxis promoted. Under a BSD style licensee future version of the software could be closed source and make mediation opaque, un-contestable and privately owned. This would limit the possibility for creative reconstructions, limit transparency and make the archives construction less contestable. The GPL or general public licensee was considered a good fit as it would preserve access to the source code for derivate work.

As Metavid begin to closely investigate the GPL in the context of online services it was discovered that network services were not considered as a form of software distribution. Since it was important that all participants in metavid have access to the source code regardless of wheatear the software was installed locally or ran over a network, we chose the aGPL (the affero general public license). This little known license is nearly identical to the GPL with the exception of online usage of aGPL code must include a link to the source. This address the issues of web services getting around the distribution of the alterations to the code base that they have made, by declaring “any user interacting with the Program [is] given the opportunity to request transmission to that user of the Program’s complete source code”. (agpl sec 2d)

1 comment dale

Freedom From Complexity

Bong Dizon has an short article focusing on the first freedom of Open Source software; the freedom to run. Dizon argues that FOSS(free open source software) developers should focus more on usability and windows support for their FOSS applications.

“It should be remembered that true software freedom is not “free as in free beer” but “free as in freedom”. Freedom from complexity is an essential and inherent part of running a computer program.”

The articles somewhat confuses the concept of complexity hiding by narrowing the definition of complexity hiding to “ease of use” GUI interface interactions, when complexity hiding is already the core development ideology for software in general. The development we do on metavid for example is dependent on huge mountains of complexity being hidden for us in software libraries command line utilities and programming environments. And what metavid does is essentially hide that complexity so that I and others can easily do complex tasks such as collectively mediate audio video streams.

What makes open source different from closed source is not that the complexity is truly hidden rather the complexity is visible upon request. This quality of open source software could more accurately be called freedom to engage complexity. Truly free software is really good at minimizing the amount of complexity necessary to be engaged at any given context.

Dizon’s article hits on is a reoccurring criticism of open source software which essentially is; open source developers are comfortable with a different level of complexity than participants who come to use that software in a different context/level of complexity. But if you look at these seemingly complex open source tools they are popular because they hide complexity very well for the context in which they are engaged. Whenever anyone chooses to engage with software they are counting on someone to have abstracted away the complexity of completing a given task, which enables them to engage at the minimal level of complexity possible to accomplish a given task. This is the same principal that governs choices among open source software systems. In the ideal Open Source development scenario, you engage with a minimal level of complexity necessary to accomplish your task and then share that solution making future engagements/enhancements that much easier.

We want software to work in as many context as possible we need only to minimize the complexity necessary to alter the software to another context. This is a shared goal with closed source software. But in close source software this complexity hiding is negotiated differently and parts of the complexity is truly hidden resulting in the bounding of applicable contexts for the software. This applicable context may limit the sharing the application with others who don’t have the economic resources to purchase the software, or the application can prevent the removal of advertisements or it may support the interests of the corporate application provider above the interests of the participant. Most of the time this is more or less a fair trade off given the context in which you’re engaging may not necessitate a different engagement level. Additionally the proprietary application provider’s context boundaries are often malleable through subversion in the form of “software piracy”, unapproved application hacking, and the hybridization of closed and open systems. In other words alternative engagements are possible from within a corporate application context provider, such as running FOSS applications on pirated copy of windows.

But again depending on the context these subversions can result in exploitive relationships whether it’s the small software company with outdated operation model being put out of business by software piracy or the large software company pirating and appropriating participatory culture to maximize their network centrality and bottom line. (*). To reduce these exploitive relationships we turn back to open source software and how the negotiation of FOSS complexity can maximize its applicability to many participant tasks and contexts without exasperating exploitive relationships.

The principal of unbounded free complexity engagement can best be illustrated in social development infrastructure for the Firefox web browser. The base browser Firefox itself is a reduction in complexity from the all in one mozilla application from which it came. Firefox underling code base (the gecko layout engine) is open source which allows to fit other contexts in the form of alternative browsers such as Camino, Netscape, Epiphany, Galeon, Flock and K-Meleon to name a few :). Firefox is easy to install and use but its real strength comes is its platform structure that lets participants engage with a minimal level of complexity to add on a feature or reduce the complexity of a given task.

The extension framework allows participants to do everything from block adds to build systems for community alterations of web site. An important element of the extension framework is that it strongly promotes the extensions being open source themselves. Essentially plain text files are copied and are read writable in your Firefox extension directory. This is essentially open source in structure, letting people build off the extensions of others.

Most importantly the extension framework supports minimal level of complexity engagement to accomplish a given task. What we could charchtzie as the essential freedom of FLOSS software to engage with the level of complexity of your choseing.

(aside reference)
*Microsoft could essentially give windows away for free and still profit off of its network centrality but is not in a position to do so given the profitability to the unquestioned centrality that it already has. Although in other countries where the applicability of their economic constraints hurt their network centrality we see much more support for free or reduced cost distribution to ensure continued network centrality in relation to other software operating systems that do not have that economic constraint. Google on the other hand has no operating system or browser network centrality and is publicly distressed about the implications. To ensure their continued network centrality this sphere goggle would probably not mind the losses of some levels of control for the potential benefits. Google already pays for searches with firefox and 1$ per Firefox download with their extension so it appears they are already adopting this strategy, it will be interesting to see if they take this to the operating system level.

Add comment dale

open roadtrip

Breaking news from our Better Late Than Never department: we sat down with Scott Shawcroft back in March to talk about Metavid for his Open Road Trip project.  He has posted a video of the interview and demo.  Check out some of his other interviews too, others from the bay area leg of his road trip include Larry Lessig and Rick & Megan Prelinger.

Add comment aphid