How do I retrieve data from an HTML page using Perl? I have tried all the suggestions on this website and have failed

advertisements

How can I extract data from an HTML page I've tried everything including suggestions on this web site, such as table extracts, but nothing works. I'm trying to extract option information from a web site that has eight headings and eight columns so I tried the suggested post:

use HTML::TableExtract;
use LWP::Simple;

my $html_string = get('http://www.barchart.com/commodityfutures/Australian_Dollar_Futures/options/A6');

$te = HTML::TableExtract->new( headers => [qw(Strike Open High Low Current Change Time     Prem ($) )] );
$te->parse($html_string) or die "Didn't find table"; 

foreach $row ($te->rows) {
    print join(',', @$row), "\n";
}

but this didn't work any help will be appreciated, thanks.


The problem is that the column headings don't match the ones in the table :) It might look like they do but, if you do:

use Data::Dumper;
$a = [qw(Strike Open High Low Current Change Time     Prem ($) )];
print Dumper($a);

You get:

$VAR1 = [
      'Strike',
      'Open',
      'High',
      'Low',
      'Current',
      'Change',
      'Time',
      'Prem',
      '($)'
    ];

Note the final two columns. What you probably need is:

$te = HTML::TableExtract->new( headers =>  [ "Strike", "Open", "High", "Low",
    "Current", "Change", "Time", "Prem (\$)" ] );