EPG grabber и Tvheadend

[adskiqt]

Търся как да добавя работещ xmltv модул за българските програми.

[turbopower]

Не съм го провил, но начина е такъв. Мога да помогна ако има нужда.

My "go" file has the following addition in it, to install the XMLTV components:


--- Code: ---# install the TVHEADEND / XMLTV package and copy the working tv_find_grabbers to /user/bin
installpkg /boot/xmltv_package/xmltv-0.5.63-i486-1_W-W.txz
cp /boot/xmltv_package/tv_find_grabbers /usr/bin/tv_find_grabbers
chmod +x /user/bin/tv_find_grabbers
/usr/bin/tv_find_grabbers

--- End code ---

This then sets up the tv_grab_file in /usr/bin and runs the tv_find_grabbers so TVHEADEND knows it exists.

I am using the Windows-based XMLTV utility which grabs my lineup from Schedules Direct and saves it to an XML file.
I downloaded the XMLTV zip and extracted it to my windows machine's C: drive.
I then configured the XMLTV program (in Windows) by running:


--- Code: ---xmltv tv_grab_na_dd --configure

--- End code ---

I set it up for my Schedules Direct lineup and then I was ready to grab the actual EPG.

The command I'm using (as I'm in Canada) is:


--- Code: ---xmltv tv_grab_na_dd --days 14 --out channels.xml

--- End code ---

This grabs 14 days of EPG data and saves it to a file called "channels.xml". This file is then copied to /mnt/extra/plugins/tvheadend/epg on my unRAID server. I created a batch file that runs this command and then scheduled a windows task to fire it every day at 5:00AM.

Here's the batch file code:

--- Code: ---REM CHANGE TO WORKING DIRECTORY
cd c:\
cd xmltv

REM REMOVE EXISTING XML FILES
del channels.xml
del \\unraidbox\xbmc\extra\plugins\tvheadend \epg\channels.xml

REM GENERATE NEW XML FILE
xmltv tv_grab_na_dd --days 14 --out channels.xml

REM COPY NEW XML TO UNRAID TVHEADEND SHARE
copy channels.xml \\unraidbox\xbmc\extra\plugins\tvheadend \epg


--- End code ---

Here's the real kicker, you have to change the 'tv_grab_file' to let it know to grab the local file:

I modified /usr/bin/tv_grab_file line '7' to read:

--- Code: ---cat /mnt/extra/plugins/tvheadend/epg/channels.xml

--- End code ---

So, the tr_grab_file now knows to look directly to my share to find the current XML file.

After I did that, when I clicked on "Save Configuration" under the XMLTV table in TVHEADEND's WEbGUI, it loaded the file, but still was not populating the EPG. I then discovered that I had to go into Channel Configuration and "point" each channel to it's EPG equivalent. This was done by clicking in the XMLTV SOURCE field in the Channels tab of the TVHEADEND webGUI. Once I did that, it extracted a list of the "discovered" channels from my channels.xml file. I matched them all up, hit "Save Configuration" again in the XMLTV GUI and blammo, populated with all kinds of data

I hope this may help you, I know it's not a totally unRAID based solution, but there appears to be no way at the present to get XMLTV running right under unRAID due to PERL dependencies that are missing. I went through weeks of agony over this and the solution I am using now is the "best case" scenario for me.

Cheers,

Km.

Това са конфиг файловете които идват с операционната система и трябва да се направи такъв и за неашите програми.

https://packages.debian.org/sid/all/xmltv-util/filelist

Това е модула на tvheadend който ги вика.

https://github.com/adamsutton/PyEPG/...abber/xmltv.py

[adskiqt]

Така е, но не се получава извеждането на ЕПГ-то в самия хеаденд, има някакъв бъг явно, иначе си направих tv_grab_bg:

#!/usr/bin/perl -w

=pod

# Инсталирате xmltv и слагате този скрипт при остаnaлите скритове от xmltv пакета.  

=cut

use strict;
use utf8;
use Encode;
use XMLTV::Configure::Writer;
use XMLTV::Options qw/ParseOptions/;
use LWP::Simple qw/$ua get/;

$ua->agent("xmltv/$XMLTV::VERSION");
my @channels_ignore = (190,84,177,168,94,147,87,121,111,169);
my @channels_ok = (93,99,116);

my %tv_attributes = (
   'source-info-name'   => 'Dnevnik',
   'source-info-url'     => 'http://www.dnevnik.bg',
   'source-data-url'     => 'http://www.dnevnik.bg/tv/',
   'generator-info-name' => 'tv_grab_bg',
  #'generator-info-url'   => 'http://...'/',
);

my ($opt, $conf) = ParseOptions ({
    grabber_name => 'tv_grab_bg',
    capabilities => [qw/baseline manualconfig apiconfig/],
    version => '$Id: tv_grab_bg,v 0.1 2009/04/16 $',
    description => 'Bulgaria (from www.dnevnik.bg/tv)',
    stage_sub => \&config_stage,
    listchannels_sub => \&list_channels,
    defaults => { days => 7, offset => 0, quiet => 0, debug => 0, },
});

# At this point, grabbing routines take over from ParseOptions

die "Error: You cannot specify --quiet with --debug, exiting"
   if ($opt->{quiet} && $opt->{debug});

if (not defined( $conf->{'channel'} )) {
   print STDERR "No channels selected in configfile " .
                $opt->{'config-file'} . "\n" .
                "Please run the grabber with --configure.\n";
   exit 1;
}

###############################################
############### GRAB THE DATA #################
###############################################

# Configure output and write XMLTV data - header, channels, listings, and footer
my $writer;
my @channels = ();
setup_xmltv_writer();
write_xmltv_header();
write_xmltv_channels($writer);
write_xmltv_programs();
write_xmltv_footer();

###############################################
################ SUBROUTINES ##################
###############################################

sub config_stage
{    
 my ( $stage, $conf ) = @_;

 my $result;
 my $writer = new XMLTV::Configure::Writer( OUTPUT => \$result,
    encoding => 'utf-8' );

 $writer->start( { grabber => 'tv_grab_bg' } );
       
 # The select-channels stage must be the last stage called
 $writer->end( 'select-channels' );

 return $result;
}

sub list_channels
{
 my $result;
 my $writer = new XMLTV::Writer(OUTPUT => \$result, encoding => 'utf-8');

 $writer->start(\%tv_attributes);

 write_xmltv_channels($writer);

 $writer->end();
 return $result;
}

# Determine options for, and create XMLTV::Writer object
sub setup_xmltv_writer
{
   # output options
   my %g_args = ();
   if (defined $opt->{output})
   {
       my $fh = new IO::File ">$opt->{output}";
       die "Error: Cannot write to '$opt->{output}', exiting" if (!$fh);
       %g_args = (OUTPUT => $fh);
   }

   $writer = new XMLTV::Writer(%g_args, encoding => 'utf-8');
}

sub write_xmltv_header
{
   $writer->start(\%tv_attributes);
}

sub write_xmltv_footer
{
   $writer->end;
}

sub write_xmltv_channels
{
 $writer = $_[0];

 my $url = "http://www.dnevnik.bg/tv/index.php";
 my $page = get $url;
 die "Couldn't get $url" unless defined $page;

 $page = encode('utf-8', $page);

# while ($page =~ m/<input type="checkbox" name="channels\[\]" value="[0-9]+" [a-z]*\s?id="prgs">[^<]*/g)
 while ($page =~ m/<input type="checkbox" id="channels[0-9]+" name="channels\[\]"  value="[0-9]+" [a-z]*\s?id="prgs">[^<]*/g)
 {
   my $channel_data = $&;

   # remove: id="channels\[0-9\]+"
   $channel_data =~ m/[0-9]+/;

   # extract channel ID
   $channel_data =~ m/[0-9]+/;
   my $channel_id = $&;

   # extract channel name
   $channel_data =~  m/[>]+[^<]*/;
   $& =~ m/[^>]+/;
   my $channel_name = $&;

   my $ignore = 0;
   my $ignored;
   foreach $ignored(@channels_ignore)
   {
     if ($ignored == $channel_id)
     {
$ignore = 1;
last;
     }
   }

#    $ignore = 1;
#    my $ch_ok;
#    foreach $ch_ok(@channels_ok)
#    {
#      if ($ch_ok == $channel_id)
#      {
# $ignore = 0;
# last;
#      }
#    }

   if ($ignore == 0)
   {
     $writer->write_channel({id => $channel_id . ".dnevnik.bg",
 'display-name' => [[ $channel_name ,"bg" ]]});
     push(@channels, $channel_id);
   }
 }
}

sub write_xmltv_program
{
 my $seconds = $_[0];
 my $url_channel = $_[1];
 my $after_midnight = $_[2];
 
 my ($sec_,$min_,$hour_,$day,$month,$yr19,@rest) =   localtime($seconds);
 ++$month;

 my $date_day = sprintf("%02d",$day);
 my $date_month = sprintf("%02d",$month);
 my $date_year = ($yr19+1900);

 my $url = "http://www.dnevnik.bg/tv/?den=" . $date_day . "/" . $date_month . "/" . $date_year . "&channels[]=" . $url_channel . "&sthour=&enhour=&tAction=submit";

 my $page = get $url;
 die "Couldn't get $url" unless defined $page;
 $page = encode('utf-8', $page);

 my $last_hour = 0;

 my @xml_start_date;
 my @xml_channel;
 my @xml_title;
 my $xml_end_date="";

 while ($page =~ m/<li><span (class="tvlistnow")?><em>[0-9:]+<\/em><\/span><h3>[^<]*/g)
 {
     my $prog_data = $&;

     $prog_data =~ m/[0-9]+/;
     my $hour = $&;

     $prog_data =~ m/:[0-9]+/;
     $& =~ m/[^:]+/;
     my $minutes = $&;

     $prog_data =~ m/<h3>[^<]*/;
     $& =~ m/>[^<]*/;
     $& =~ m/[^>]+/;
     my $title = $&;

     if ($last_hour > ($hour+0))
     {
#icrement day
$seconds = $seconds + 86400;
my ($sec_,$min_,$hour_,$day,$month,$yr19,@rest) =   localtime($seconds);
++$month;

$date_day = sprintf("%02d",$day);
$date_month = sprintf("%02d",$month);
$date_year = ($yr19+1900);

if ($after_midnight == 0)
{
 $xml_end_date = $date_year . $date_month . $date_day . $hour . $minutes . " +0200";
 last;
}

$after_midnight = 0;
     }
     else
     {
if ($after_midnight == 1)
{
 $last_hour = $hour+0;
 next;
}
     }

     $last_hour = $hour+0;

     push(@xml_start_date, $date_year . $date_month . $date_day . $hour . $minutes . " +0200");
     push(@xml_channel, $url_channel . ".dnevnik.bg");
     push(@xml_title, $title);
 }
 
 my $count = @xml_start_date;
 my $i = 0;
 for ($i = 0; $i < $count; $i++)
 {
   if (($i == ($count - 1)) && ($xml_end_date ne ""))
   {
     $writer->write_programme({  start => $xml_start_date[$i],
 stop => $xml_end_date,
 channel => $xml_channel[$i],
 title => [[$xml_title[$i], 'bg']] });
   }
   else
   {
     $writer->write_programme({  start => $xml_start_date[$i],
 channel => $xml_channel[$i],
 title => [[$xml_title[$i], 'bg']] });
   }
 }
}

sub write_xmltv_programs
{
 my $mytime = time;
 my $days = 7;
 my $offset = 0;
 my $i = 0;

 if (defined $opt->{offset})
 {
   $offset = $opt->{offset};
 }

 if (defined $opt->{days})
 {
   $days = $opt->{days};
 }

 $days = $days + $offset;
 for ($i = $offset; $i < $days; $i++)
 {
   if (($i < 0) || ($i > 10))
   {
     last;
   }

   my $channel;
   foreach $channel(@channels)
   {
     write_xmltv_program(($mytime + ($i - 1)*86400), $channel, 1);
     write_xmltv_program(($mytime + $i*86400), $channel, 0);
   }
  }
} 

Направих си настройки на самия хеаденд да търси епг-то и до там, няма извеждане.
Както и да е, мисля, че реших засега проблема кардинално, свалям каналите с астра, от там по ип в хеаденда и взе, че излезна епг-то.

[turbopower]

Изходният xml file със същите тагове ли е както генерираният, с конфигурациите по подразбиране?

Следам и URL при теб не е правилно. Частта /sled5 при теб липсва.
http://www.dnevnik.bg/sled5/tv/index...tAction=submit

[adskiqt]

Да всичко е точно в адреса, автоматично пренасочва sled5, пробвах го и по двата начина, няма проблем. Извадих информацията с епг-то, не е това проблема, проблема си е в хеаденда.
Интересното е, че само от астра към хеаденда извежда епг, преди използвах мумудвб, като първи избор, със старата(стейбъл 3,4) версия на хеаденда, няма проблем с грабера, но в новите верии може да се добавят и уникаст стриймове, с допълнитлни опции, които са идеални за размяна с мониторинг на стриймове.