On occasion we’re asked to acquire data from a site that already offers an API. In almost all cases it’s going to be simpler to acquire data from a site by accessing an API as opposed to crawling. In theory, there should be no need to scrape data from a site if an API to the content is already made available. That said, there are a number of reasons why it still may make sense to scrape a site that also provides an API.
Miscellaneous
Scraping data from various industries
We’ve just added several new scraping sessions that exemplify extracting data from sites in various industries. If you go to our home page and click on one of the buttons corresponding to an industry you’ll be taken to a page where you can download the scraping session. The e-commerce section also has a video to … Read moreScraping data from various industries
End-of-year sale!
This is our biggest sale in quite a while. Until December 31, 2012 take 40% off Professional Edition licenses and 60% off Enterprise Edition licenses. Click here to take advantage.
Boo-Boo
I made a boo-boo. To make it short, if you downloaded an installer from our web site between May 3 and May 5 you may actually be running version 5.0 of screen-scraper even though it says 5.5. This was a result of an oversight on my part, and my apologies to those affected. Fortunately for … Read moreBoo-Boo
Screen-Scraping for iPhone, Andriod, Blackberry, and Most Any Other Mobile Device
The Mobile Problem The proliferation of mobile devices has created a problem. Most web sites these days are designed to be viewed on desktop computers with high-resolution monitors and via web browsers that allow for sophisticated interactivity. Anyone who’s tried to view such sites on mobile devices with small screens can attest to a cramped … Read moreScreen-Scraping for iPhone, Andriod, Blackberry, and Most Any Other Mobile Device
Using Client Certificates with screen-scraper
We recently added experimental support to screen-scraper for client/PKI certificates. Some web sites require that you supply a client certificate, that you would have previously been given, in order to access them. I say this new feature is “experimental” because we’ve only been able to perform limited testing with it. So far, it does seem … Read moreUsing Client Certificates with screen-scraper
Anonymization Service Price Change
A few months ago we announced a drop in the price of our anonymization service for those using the latest alpha versions of screen-scraper. Unfortunately, things didn’t work out with this quite as we had planned, so, for the time being, we’re returning to our previous price of 25 cents per server per hour. For … Read moreAnonymization Service Price Change
Money-Back Guarantee for screen-scraper
We’ve actually had a 30-day money-back guarantee since almost the beginning, but recently decided to highlight it a bit more. Any time in a software development project you incorporate a new library or application you’re incurring a certain amount of risk. The new application may appear to do just what you want it to, but … Read moreMoney-Back Guarantee for screen-scraper
screen-scraper.com in the Wall Street Journal
On September 5th, 2010 we received a call from a reporter from the Wall Street Journal, Steve Stecklow (2007 Pulitzer Prize winner). He was calling to speak with someone at our company for a story he was doing related to our industry. He and I talked for about 40 minutes where he asked a lot … Read morescreen-scraper.com in the Wall Street Journal
Oracle/screen-scraper Podcast
Here I go tooting our horn again–Oracle has just posted a podcast on integrating screen-scraper with their Oracle Secure Enterprise Search product. The page that links to the podcast is here, and the MP3 file of the same can be found here. Might be worth a listen if only to get a better sense for … Read moreOracle/screen-scraper Podcast