Open Data Project

  • Open Knowledge India is fueled by the thousands of citizen activists all over the world, who devote their time and energy to the common dream of a better informed world. In India, there is a lot of scope for involvement of the masses in order to work towards making ourselves a better managed nation. Constitutional instruments like the Public Interest Litigation (PIL), the Right To Information (RTI), etc., lose their relevance without complete transparency in the dissemination of government as well as non-government data. Building a mass-based  system of knowledge-banks can help us a lot in achieving our goal. We can assist the governments for making more information public, wherever necessary.

And the list is growing…

open Census images7720images7710  

Open Data Day Talk

  • Relation Between Medieval Renaissance and Today’s Open Knowledge Dissemination

The advent of the printing press helped a lot in the democratization of knowledge and research in Medieval Europe. The arrival of the movable type printing press introduced the era of true mass communication in the western world and this, in turn, altered the fabric of the society forever. It transformed people’s lives by changing their relationship to information and knowledge. It also transformed their relationship with existing authority. It created a new breed of free thinkers, who ultimately dislodged the Dark Ages and brought about the Modern Era. Involving the ‘general public’ and saving knowledge from the hands of a select few brought about a positive change in the human society. Print created the possibility of wide and rapid circulation of ideas. This opened up a new free world where ideas could be debated and discussed. Thus the foundation of the modern scientific fervour was laid.

Print brought the ideas of thinkers, philosophers and scientists closer to the common man and made new ideas more accessible. It is a matter of debate how thinkers, like Issac Newton, could influence the thinking of the society without access to print. It would be superfluous to mention how true scientific debate started only after the invention of the printing press. The writings of thinkers outside the central sphere of science (like Thomas Paine, Voltaire, Jean Jacques Rousseau, etc.) too went a long way in shaping the world as we know it today.

However, in their time, the first printing presses were mostly viewed with skepticism. Barring a handful, most influential people opposed it. The ‘holier than thou’ expressed their fears that the world would come to and end if they let the common people use print. Even those, who welcomed the print, were apprehensive of the ‘ill’ effects that wider circulation of ideas through print could have on the world. Rulers and religious heads feared that if they did not control the printed material, their authority would vastly diminish. They wanted to have total control over what was to be printed.

In the words of Erasmus, a Latin scholar and a Catholic reformer, ‘To what corner of the world do they not fly, these swarms of new books? It may be that one here and one there contributes something worth knowing, but the very multitude of them is hurtful to scholarship, because it creates a glut, and even in good things, satiety is most harmful…(printers) fill the world with books, not just trifling things (such as I write, perhaps), but stupid, ignorant, slanderous, scandalous, raving, irreligious and seditious books, and the number of them is such that even the valuable publications lose their value.’

Erasmus’s fear pretty much sums up the apprehensions of today’s ‘intellectual elites’ and ‘printing powerhouses’. In todays world too, the Internet has brought about a transformation of the society. This is a tool that can be used for free dissemination of knowledge and of research. However, a few people even today tend to believe that free dissemination of research (that results in free and fair debates and discussions of the works) would bring about a ‘end of the world’ situation for science. They are of the opinion that they ought to have as much control as possible over the dissemination of research works in order to keep the flag of science flying. These handful of people have the audacity to believe that they must be the ‘chosen ones’ to boss over the whole of the scientific community. These are the people who oppose Open Science and Open Knowledge movements.

Time has shown us how zero control by a handful of individuals over the society and complete control of the community, as a whole, over itself brings about positive changes. Less the control by individuals or groups and more the control of the complete set of individuals, more is the positive change. The history of the printing press is a case in point. While history made a mockery of the control-freaks, it proved right the few individuals, who believed in the intellectual capacity of the masses. Intellectual Nazism should be a thing of the past and we should move away from such self-defeating practices.

As more and more researchers embrace Open practices, irrespective of the influence of any kind of authority and affiliations, a new free world of debate and discussions will truly open up. Positive Open Science movements have shown us how researchers can jump over all traditional deterrents and contribute positively towards the development of the scientific community, as a whole. Dissemination of research in Open Science Platforms (where researchers can freely upload their works) should be enough in terms of publication of research. The whole community of researchers will deliberate over these published works and not just a select few. This will take scientific dialogue forward in the true sense. As printed books replaced handwritten manuscripts, so will such Open platforms of knowledge dissemination eventually replace other non-Open pathways of dispersing knowledge.

  • What is Open Data?

“Open data is data that can be freely used, reused and redistributed by anyone – subject only, at most, to the requirement to attribute and sharealike.”

The Open Definition sets out in detail the requirements for ‘openness’ in relation to content and data. The key features are:

Availability and Access: the data must be available as a whole and at no more than a reasonable reproduction cost, preferably by downloading over the internet. The data must also be available in a convenient and modifiable form.

Reuse and Redistribution: the data must be provided under terms that permit reuse and redistribution including the intermixing with other datasets. The data must be machine-readable.

Universal Participation: everyone must be able to use, reuse and redistribute – there should be no discrimination against fields of endeavour or against persons or groups. For example, ‘non-commercial’ restrictions that would prevent ‘commercial’ use, or restrictions of use for certain purposes (e.g. only in education), are not allowed.

  • Why Open Data?

Why should data be open? The answer, of course, depends somewhat on the type of data. However, there are common reasons such as:

Transparency. In a well-functioning, democratic society citizens need to know what their government is doing. To do that, they must be able freely to access government data and information and to share that information with other citizens. Transparency isn’t just about access, it is also about sharing and reuse — often, to understand material it needs to be analyzed and visualized and this requires that the material be open so that it can be freely used and reused.

Releasing social and commercial value. In a digital age, data is a key resource for social and commercial activities. Everything from finding your local post office to building a search engine requires access to data, much of which is created or held by government. By opening up data, government can help drive the creation of innovative business and services that deliver social and commercial value.

Participation and engagement – participatory governance or for business and organizations engaging with your users and audience. Much of the time citizens are only able to engage with their own governance sporadically — maybe just at an election every 4 or 5 years. By opening up data, citizens are enabled to be much more directly informed and involved in decision-making. This is more than transparency: it’s about making a full “read/write” society, not just about knowing what is happening in the process of governance but being able to contribute to it.

  • How to Open Up Data

If you are looking for practical, more detailed, advice on how to open up data, have a look at the Open Data Handbook. The handbook discusses the legal, social and technical aspects of how to open up data. Read more in the Open Data Handbook. Here we provide some short suggestions for initial steps.

3 Key Rules

There are three key rules we recommend following when opening up data:
  • Keep it simple.Start out small, simple and fast. There is no requirement that every dataset must be made open right now. Starting out by opening up just one dataset, or even one part of a large dataset, is fine — of course, the more datasets you can open up the better.Remember this is about innovation. Moving as rapidly as possible is good because it means you can build momentum and learn from experience — innovation is as much about failure as success and not every dataset will be useful.
  • Engage early and engage often.Engage with actual and potential users and re-users of the data as early and as often as you can, be they citizens, businesses or developers. This will ensure that the next iteration of your service is as relevant as it can be.It is essential to bear in mind that much of the data will not reach ultimate users directly, but rather via ‘info-mediaries’. These are the people who take the data and transform or remix it to be presented. For example, most of us don’t want or need a large database of GPS coordinates, we would much prefer a map. Thus, engage with info-mediaries first. They will re-use and re-purpose the material.
  • Address common fears and misunderstandings. This is especially important if you are working with or within large institutions such as government. When opening up data you will encounter plenty of questions and fears. It is important to (a) identify the most important ones and (b) address them at as early a stage as possible.

The Four Steps

These are in very approximate order – many of the steps can be done simultaneously.
  1. Choose your dataset(s). Choose the dataset(s) you plan to make open. Keep in mind that you can (and may need to) return to this step if you encounter problems at a later stage.

  2. Apply an open license.

    1. Determine what intellectual property rights exist in the data.

    2. Apply a suitable ‘open’ license that licenses all of these rights and supports the definition of openness discussed in the section above on ‘What Open Data’

    3. NB: if you can’t do this go back to step 1 and try a different dataset.

  3. Make the data available – in bulk and in a useful format. You may also wish to consider alternative ways of making it available such as via an API.

  4. Make it discoverable – post on the web and perhaps organize a central catalog to list your open datasets.
  • A few Open Data projects run by the Open Knowledge Foundation :
  • BubbleTree is a library for interactive visualization of hierarchical data. Originally developed mainly for spending data, the library is now completely independent from the OpenSpending platform.
  • Datahub provides free access to many of CKAN’s core features, letting you search for data, register published datasets, create and manage groups of datasets, and get updates from datasets and groups you’re interested in. You can use the web interface or, if you are a programmer needing to connect the Datahub with another app, the CKAN API.
  • OKFN Data  :  ‘Our Mission is a World of Frictionless Data. Our immediate goal is to make it as simple as possible to get the data you want into the tool of your choice.
  • Listify : Turn a Google spreadsheet into a beautiful, searchable listing in seconds.
  • Timeliner : Make elegant timelines and timemaps using Google Spreadsheets.
  • Open Spending ‘Our aim is to track every government financial transaction across the world and present it in useful and engaging forms for everyone from a school-child to a data geek.’
  • Nomenklatura is a simple service that makes it easy to maintain a canonical list of entities such as persons, companies or event streets and to match messy input, such as their names against that canonical list – for example, matching Acme Widgets, Acme Widgets Inc and Acme Widgets Incorporated to the canonical “Acme Widgets”.
  • Annotator is an open-source JavaScript library and tool that can be added to any webpage to make it annotatable. Annotations can have comments, tags, users and more. Moreover, the Annotator is designed for easy extensibility so its a cinch to add a new feature or behaviour.
  • Kartograph is a simple and lightweight framework for building interactive map applications without Google Maps or any other mapping service. It was created with the needs of designers and data journalists in mind. Actually, Kartograph is two libraries. One generates beautiful & compact SVG maps; the other helps you to create interactive maps that run across all major browsers.
  • Recline : A simple but powerful library for building data applications in pure Javascript and HTML.
  • School of Data works to empower civil society organizations, journalists and citizens with the skills they need to use data effectively in their efforts to create more equitable and effective societies. There is huge potential to use data and open data to improve the lives of citizens around the world, especially in increasing the transparency and accountability of government. However, many of the groups who are closest to the problems – NGOs, journalists, and citizens – currently lack the skills to use data effectively — and even an awareness of the potential of data for their work. School of Data’s mission is to teach people how to gain powerful insights and create compelling stories using data.
  • Explorer: Load data from Google Docs, CSVs, DataHub, GitHub and more. Explore the data in a rich tabular interface, create graphs and maps

2 thoughts on “Open Data Project”

Leave a Reply

Your email address will not be published. Required fields are marked *