One reason i-banks and academics pay top dollar for data is reliability and consistency. How does Data Marketplace address errors in user-collected data? Do they guarantee its quality? If I spend $10,000 on a quarterly update on CRSP and find a systematic mistake, they will fix it without charge. Such mistakes are hard to find because CRSP has a reputation to maintain. I wonder if users hired to collect data have the same constraints.
We want to be as transparent as possible about who is selling the data. Ultimately, it will be the reputation of the seller that determines how 'trustworthy' the data is (and how much you should be willing to pay for it).
The name of the company is Data Marketplace i.e. a marketplace for multiple vendors. I think their goal is going to be to aggregate the various providers of data and through competition, quality should be relatively stable and hopefully improve. I'd compare it to buying a ticket through orbitz. There might be a little bit more customer service overhead if you have a problem with your ticket (or data), but the tradeoff is you don't have to shop at 20 different places to get what you want, plus the price might be more competitive.
Having worked in a data management group at a large financial institution I can say there seemed to be little relation between price and reliability and consistency. There were whole teams devoted to standardizing and checking data for errors and maintaining history, not to mention regenerating datasets whenever historical corrections were made. Pricing was mainly determined by who owned the rights to the data and how much in demand it was.
But demand -- in the long run -- is directly related to reliability/consistency. I think that this model can work, but it will take time before institutions integrate the data into mission-critical applications. I see this idea as a means of small, niche data-providers to start their own firms and perhaps start to provide the data independently of Data Marketplace.
Abundance of data sources and collectors can usually increase data quality. For instance, when we post something on mechanical turk we can ask multiple users to repeat the same task thereby scrubbing the output better.
(I also wish TC crowd-sourced grammar checking of their articles prior to pushing them out:-) )
Hey, it sounds like these guys would be a perfect customer for ITA's Needle. Needle provides a GUI-based way for nonprogrammers to scrape data, clean it up, and get it into a structured database where they can create views on it and all that databasey stuff. http://www.needlebase.com/
I watched an internal demo :) Where they scraped a table off an MLB site, live, using their GUI and prettified the data a bit. If you check the blog there's some demonstrations, and the Village Voice gave it a test drive: http://blogs.villagevoice.com/music/archives/2010/01/pazz_jo...
I'm really interested in what would make it really awesome for you. In particular, is the cleanup/database part important for you? If you or anyone on HN wants to talk about it (and especially if you've tried http://siteparse.com), my email is in my profile.
What we are doing is not really ready to launch, but we match the first part of this description perfectly ("a GUI-based way for nonprogrammers to scrape data"). The GUI is web based, you can watch a demo and try us out immediately (you get your data in CSV):
How funny! Right as I start talking about my startup to buy and sell datasets, it turns out YC had something like this in secret the whole time. At least I know I'm doing something worth while.
Let's talk! I like where you are going with Webscaled-- maybe we could work together. Shoot me an email me when you get a chance: matt @ datamarketplace.com
As a pay-for-service approach, it seems reasonable: you say you need some data, the appropriate people come up with it in the format you want, and you pay and get the data. But the Amazon-like angle seems odder to me: How will each of the bits of structured information manage to be sold more than once? Since most isn't copyrightable, anyone can buy a copy and then put it up online for free. I could go buy that Wal-Mart Store Locations data set for $30 right now, and publish it on my blog. Are you hoping EULAs will be enough to prevent this, or just that in practice it won't happen enough to matter, or people won't notice that the data is free online and continue buying it anyway?
This is an interesting question that we've been giving a lot of thought to lately. We hope that it won't be a big problem, but if it is we will need to implement public EULAs and a dispute resolution process.
In terms of multiple purchases, we suspect there will be some datasets that are more popular than others. A list of Wal-Mart stores for example could be useful to a number of different people, such as investors, real estate developers, business planners, marketers, etc.
I think you guys are doing the right thing by not worrying about EULAs and other protection mechanisms at this stage in the game. Stay focused on satisfying your customers and improving the product as much as possible until you start making major headway.
Is it just me, or are more YC companies launching early this cycle? I can count 5--Etacts, Crocodoc, Cardpool, 140bets, and Data Marketplace--that have already launched.
Is this just a product of them having more companies this cycle, something they're stressing this year, or just a coincidence?
Yeah, it's true. We definitely felt like we were playing catch-up from day 1. Not that its a competition, but I think companies benefit more from YC having a working product earlier on.
It makes sense that if YC is getting more applications than their applicants will be better and more prepared. Several companies in this round had already produced several products, were ramen profitable, and had been operating for sometimes years previous to becoming a YC company.
They will still get lots of help. Correct me if I am wrong. I think the biggest advantage of being a YC company right now is the kind of press, reputation, and trust of investors founders might get. PG's advice is certainly valuable but their connections are the biggest deal.
lots of groups switch ideas mid-session and have to start from scratch on day >1. but yes, launching early (more specifically before demo day) is quite an advantage.
I didn't see it posted anywhere, but on a $4 sale, I received $3.68, so that's an 8% fee if it's a flat rate. (Or it could be something like $0.28 + 1%)
@matthodan, you might want to post that somewhere / make it more obvious.
I'm getting an error when trying to upload a CSV data set. :( Not very encouraging to be prompted with an error at my first attempt to use the product. Maybe TC jumped on the story a little soon?
Sorry to hear that. Would you mind emailing me the CSV file at steve@datamarketplace.com? I'll try and recreate the problem and let you know when it's fixed.
Seems like an awfully low signal to noise ratio in the requests at the moment. The vast majority of them are very unrealistic, in not outright spam.
"Request For Data
List of all Assisted Living care centers in the U.S., including name, address, phone number, website and any photos on their website. There should be about 20,000 of them.
Budget: $20.00
Deadline: March 25, 2010 10:40 PM"
This looks really interesting, though if one is looking for publicly available data, I think it's an open question of how much of the value is in providing a marketplace vs. actually doing the aggregation, organization and some analysis.
Two other interesting startups in this space that are focussed more on the data aggregation end are http://www.AnythingResearch.com and http://www.AggData.com (which has dozens of retail datasets just like the Walmart store example, presumably gathered by screen-scraping).
Data Marketplace can help people who are looking for data connect with people who are able to collect it. One of the problems with all of the data online is that it is in too raw a format to be useful. Someone must scrape/combine/clean the data before it can be analyzed. By connecting business people who maybe don't know how to scrape data with programmers/aggregators who specialize in it, we hope to make it a lot easier to find useful data.
Pricing is set by the provider, but we don't mind haggling. We're trying to build a community around buying and selling data tables, so if the price isn't right, that's good feedback for the seller.
I'm just about finished with a dissertation in Finance (teaching job starts in August). Obviously I'm very excited about this launch, I have a couple painfully acquired datasets I wouldn't mind seeing a dollar return from and it'd be great to save some effort going forward, especially after I have a salary to spend on that text box.
We'd love to help you get started! If you have any trouble adding your datasets, please send me an email at matt @ datamarketplace.com and I'll make sure they get added asap.
I would love the ability to browse available datasets without having to register. And I'd really love to be able to request a dataset without having to register.
Structured datasets are hard, and the hardest part is finding one that updates as time passes and new data is available - how are y'all handling that?
You should be able to browse the data that is available without registering by clicking on "Buy Data" in the header. We make people register when they request a dataset because we want to be able to let them know once we've found their data.
We're working on an API that will allow sellers to programmatically refresh their dataset to keep it up-to-date with the source. We're also working on a search and filtering system that ranks datasets based on when they were last updated.
Ah, OK, I thought that was just a representative sample since there were only five datasets.
I'd recommend you find, package, and sell other datasets that exist out there (Freebase, etc.).
I disagree with the registration requirement - if I want to be updated, make that a opt-in box - but I acknowledge that it's a justifiable product decision.
This is every grad student researcher's dream... data finding and cleaning is such a major hassle. If only I had this a couple years ago...
This is going to be great especially when the new Census comes out. Lots of data to be sifted through and organized in unique ways that the government never does for us.
No problem. I think you can get A LOT of data from grad students. They hold the keys to a lot of it, and aren't held to some contract to never release it.
I would redesign the site to be more enterprise like than web-application-like. This is more b2b. I think I am going to buy or sell some data on this site.
