As we crawl and index the world’s data silos, we’ve noticed that some have APIs, to this extent we’ve gone ahead and created an API wrapper for those data sets that are already accessible in a machine readable format.
The benefit of this is that you’re now able to build on top of both NYC data and Chicago data through the Exversion API without having to go to either city’s open data site.
While the functionality is there, some problems still persist, primarily that we’re limited to the API functionality of the host website, and henceforth some API functions may not be available.
For example, while you can query a data set housed on Exversion to give you all results that have a single word in a column, the same cannot be said for external data we’ve wrapped from places such as NYC, San Francisco, or Chicago, where you’ll need to query a full text.
Which is why we urge you to download the data set and upload it to the platform to take full advantage of our API, and you can do this easily, by clicking the Menu box and selecting download and then CSV, and then clicking the Green Upload button. When you do this, the meta, description, and data attribution fields will be automatically populated. Just add a blurb describing the changes (if any), and upload the file.
For the above data set, City of Chicago / Christmas Tree Drop of Locations, the entire process took approximately a minute, making the data readily consumable by anyone on the site, and when you or anyone else searches for the data on Exversion, because it’s now hosted, it will appear at the top of your search.
At the end of June we added metadata for over 20,000 data.gov data sets on Exversion, and are in the process of adding thousands of other data sets that are housed on CKAN installations throughout the world.
However, the fundamental problem with CKAN meta data aggregation however is that the CKAN API will not let you query against the actual data set, but instead query what types of data sets there are on each installation of the software, making the platform useless in terms of machine readable data.
This model distributes static .csv files along with secondary and tertiary links to off-site data, making it very difficult to aggregate much of anything aside from the link / meta data. As such, we’re asking you, the crowd, to help populate these data sets. In order to foster this process we’ve provided a simple applet that will allow you to upload a specific data set.
In the above example, you see that I’ve searched for Crash Statistics by state on Exversion, but the data set is yet to have been imported, namely it needs to be cleaned. Following our style guide, I quickly made the dataset “Exversion ready” i.e. a CSV file, and uploaded to the site with a description of the changes, making the dataset is then available here.
While this is not the perfect solution to the larger problem of not having easily accessible machine readable data, it allows the data community to come together and help make data that has been previously inaccessible, machine readable.
At the same time, while this is the status quo for data housed / linked to on CKAN installations, we’re working on a few projects that should wholly integrate data housed on other platforms.
If you guys have any questions ask them in the comments of feel free to write us at info @ exversion.com