Multithreading – Perl threads separate themselves

I'm new to Perl (and programming, too) and have been playing with threads for the past few weeks. So far, I understand that it's frustrating to use them to perform some similar parallel tasks - if you have a number of threads, memory consumption is uncontrollable, depending on some input values, and it seems foolish to simply limit that number and make some temporary connections

#!/usr/bin/perl
#

use strict;
use warnings;
use threads;
use NetAddr::IP;
use Net::Ping;
use Thread::Queue;
use Thread::Semaphore;
########## get my IPs from CIDR-notation #############
my @ips;
for my $cidr (@ARGV) {
    my $n = NetAddr::IP->new($cidr);
    foreach ( @{ $n->hostenumref } ) {
        push @ips,( split( '/',$_ ) )[0];
    }
}

my $ping      = Net::Ping->new("icmp");
my $pq        = Thread::Queue->new( @ips,undef );    # ping-worker-queue
my $rq        = Thread::Queue->new();                 # response queue
my $semaphore = Thread::Semaphore->new(100);          # I hoped this may be usefull to limit # of concurrent threads

while ( my $phost = $pq->dequeue() ) {
    $semaphore->down();
    threads->create( { 'stack_size' => 32 * 4096 },\&ping_th,$phost );
}

sub ping_th {
    $rq->enqueue( $_[0] ) if $ping->ping( $_[0],1 );
    $semaphore->up();
    threads->detach();
}

$rq->enqueue(undef);

while ( my $alive_ip = $rq->dequeue() ) {
    print $alive_ip,"\n";
}

I can't find a comprehensive description of how thread - > detach() should work in a thread subroutine and think it might be useful... And it does - if I do something in the main program (thread) to extend its life cycle (sleep does well), so all separate threads complete and drain their parts into my $RQ, Otherwise, it will run some threads to collect its results to the queue and exit the warning, such as:

Perl exited with active threads:
    5 running and unjoined
    0 finished and unjoined
    0 running and detached

Making the main program "sleep" for a while seems stupid again - is there a way for threads to complete their things and separate only after the actual thread - > detach() call? So far, my guess is that once a thread is created, threads - > detach () in the child thread will be applied, so this is not a method I use the old version of centoss V5 10.1 tried this This should change modern V5 16 or V5 18(usethreads-compiled)？

Solution

Separating a post is not particularly useful because you effectively say "I don't care when they quit."

This is usually not what you want - your process completes while the thread is still running

Typically – creating a thread incurs overhead because your process is cloned in memory You want to avoid it Thread:: queue is also useful because it is a thread - safe way to pass information In your code, you don't actually need $PQ because you don't actually thread it when you use it

Your semaphore is one way to do this, but I can suggest an alternative:

#!/usr/bin/perl
use strict;
use warnings;
use Thread::Queue;

my $nthreads = 100;

my $ping_q = Thread::Queue -> new(); 
my $result_q = Thread::Queue -> new(); 

sub ping_host {
     my $pinger = Net::Ping->new("icmp");
     while ( my $hostname = $ping_q -> dequeue() ) {
         if ( $pinger -> ping ( $hostname,1 ) ) { 
              $result_q -> enqueue ( $hostname ); 
         }
     }
}

#start the threads

for ( 1..$nthreads ) {
     threads -> create ( \&ping_host );
}

#queue the workload
$ping_q -> enqueue ( @ip_list );

#close the queue,so '$ping_q -> dequeue' returns undef,breaking the while loop. 

$ping_q -> end();

#wait for pingers to finish.
foreach my $thr ( threads -> list() ) {
   $thr -> join();
}
$results_q -> end();

#collate results
while ( my $successful_host = $results_q -> dequeue_nb() ) {
    print $successful_host,"\n"; 
}

In this way, you can generate threads in advance, queue the target, and then sort out the results after completion You will not incur the overhead of repeatedly regenerating threads, and your program will wait for all threads to complete This may be a while, because the Ping timeout on the 'down' host will last for some time