I recently got back into coding after a 4 - 5 yr hiatus of sorts.

I mainly developed in C/C++ and C# but have recently decided to give Java a look mainly due to it being the language of choice for Android development.

My first project to kind of learn the language is an application to scan ebay auctions for items that have bids on them and thus demonstrate a level of interest.

I;m guessing I'll then throw the auctions into a MySQL database for analyzing.

I was thinking maybe i could use it to find similar items and generate a price history for these items and maybe identify deals.

Or even something as basic as just identifying what sells.

I don't know if there is commercial or free open source projects like this available but this is more for my own learning so that I can pick up JAVA which I imagine will be relatively easy with my background in C#.

Anyone ever done anything kind of application like this for ebay?

I have not but i believe i read some where that ebay has exposed an xml api for people to use in their applications.

i did a website like that a few years back just for fun, it never went online.

ebay has a pretty powerful api with which you can do exactly what you want: http://developer.ebay.com/common/api/
there is also a java sdk: http://developer.ebay.com/developercenter/java/sdk/

all this is free as far as i know; at least it was back in the day.

Hi,

Thanks for the reply.
I initially looked into the API that ebay exposes and for some reason I could of swore it charges you for a licence and per call made to the api but on second glance ....it does appear to be free for a certain amount of calls.

Would of saved me a lot of work parsing their html :S

So I decided against using the Ebay API and just parsed the html instead.

As of now I just dump all of the auctions with bids into a database. My plan is to be able to use the data to potentially generate price history for items so that maybe I can have it identify deals / low priced items, etc.

Any ideas on how I can identify identical items based on the ebay item descriptions supplied by the sellers?

As of now my code just connects to ebay and retrieve auction listing based on a search result ie: "Wayne gretzky rookie card" but this is hardly fool proof and I end up with some aucions which aren't gretzky rookie cards.

For example I might get an auction for a gretzky reprint rookie card which is like $2.99 compared to the real gretzky authentic rookie which is $500+ and this skews my price history graph and generates false alerts.

I imagine the only way to filter this is by manually reviewing the data..or possibly incorporating a black list of words like "reprint" and just not record that data..idk, any ideas?

no naive approach is going to work here, you have to know a little bit about your subject and probably have to customize your approach for every different item. So building in flexibility is the most important thing. The blacklist is a sensible idea, to make it as easy as possible to specify a blacklist for each search you could accept something like "wayne gretsky rookie card -reprint".

If you know the expected price is 500, you could tell it to discard results outside a reasonable price range (can you estimate a standard deviation?) You have to be careful here, because you are looking for outliers (bargains), but I take it getting an actual Wayne Gretsky rookie card here for $2.99 is astronomically unlikely.

Finally, if you want a super scalable, robust and fully automated way to do it, you should ask yourself, what would Google do? Google would start with a fairly basic product and get their customers or beta testers to flag the results that aren't what they were looking for. Then they would throw all the results in a database and use some neural net or decision tree-building algorithm to predict exactly which listings match a given search query. After a few hundred thousand uses, it should be 99% accurate. I don't think this is really what you're after, though.

And in this case, I notice eBay has a separate category for "reprint" that you can search for when using the site. This might be hard to scrape whatever way you are doing it, but I bet you could get at it really easily with the API.