Thoughts, tips, and updates on screen-scraping
Todd Wilson The holiday enhancements have spilled over into 2012: Added “Always at the end” option to force scripts to run at the end of a scraping session, even if it gets stopped prematurely. The prompt to save dialog box only shows on exit when a change has actually been made. Added a keyboard shortcut to the … Read moreVersion 5.5.33a of screen-scraper Released
Things have cooled down for us a bit over the holidays, so we’ve been able to carve out time for a number of bug fixes and feature enhancements. Here’s the list: Fixed a threading issue related to the REST interface. Added classes and methods related to decoding images. Fixed a bug related to use of … Read moreVersion 5.5.32a of screen-scraper Released
Most of the time when extracting information from web sites you’ll deal with HTML, which is generally pretty straightforward to deal with. Occasionally, though, content will be delivered via something like a Java applet or Flash movie. Just recently I completed a project that dealt with extracting data from a Flash movie, where the data … Read moreScraping AMF Sites
A few fixes in this release: Fixed a bug that was causing the user-agent header to be duplicated. Fixed a bug where a deleted recent script still shows in the script drop-down list. Fixed a bug related to multi-exports.
Just a few changes: Deprecated caching and filtering data sets (can be re-enabled with EnableCachingAndFilteringDataSets property). Now automatically swapping extractor pattern tokens for embedded variables in certain fields in the workbench (e.g., in the URL field ~@FOO@~ is changed to ~#FOO#~). Added a “Find” button to the “Last Request” tab.
Get ready, kiddies, this is a big one! Found myself with some time on my hands, so I got some things done that have needed doing for a while. Plus I added in a few little goodies that have been rolling around in my head. Enjoy! Now outputting message as a warning when extractor pattern … Read moreVersion 5.5.23a of screen-scraper Released
Several fixes and enhancements in this one: Fixed a bug where a null parameter was causing rendering problems. Added ability to turn on and off automatic proxy cycling via setAutomaticProxyCycling. Auto-saving can now be enabled by adding an AutoSaveTime=[Time in seconds] in the screen-scraper.properties file. Filtered data sets now show up as filtered when using … Read moreVersion 5.5.17a of screen-scraper Released
Just a small fix in this one, but definitely a recommended upgrade for those using alpha versions. This release fixes a bug introduced in the previous alpha related to closing the HTTP connection manager.
This one contains a few important fixes. For those using alpha versions I’d recommend upgrading. Here’s what it contains: Fixed an issue where a blank file HTTP parameter was being sent incorrectly. Fixed an issue with the REST interface where the wrong scrapeable_session_id was being returned. Fixed an issue where the HTTP connection manager was … Read moreVersion 5.5.14a of screen-scraper Released
Several minor bug fixes and updates in this one: Fixed a race condition where a scraping session could potentially get started by two different threads. DataManager: a few logging changes DataManager: a modification of the order of database writes when foreign keys are manually set DataManager: transactional support for rolling back writes DataManager: a framework … Read moreVersion 5.5.12a of screen-scraper Released