Multithreading – Perl multithreaded programs occasionally crash

I wrote a program using multithreading in Perl I am using this program to understand how multithreading is implemented in Perl

First, briefly describe what the program is going to do: it will read a list of URLs from a text file, one at a time For each URL, it calls the subroutine (passing the URL as a parameter) and sends it an HTTP head request Once the HTTP response header is received, it prints the "server header" field from the response

For each URL, it starts a new thread that calls the above subroutine

Problem: the main problem is that the program sometimes crashes intermittently It works normally at other times It seems to be unreliable code, and I believe there is a way to make it work reliably

code:

#!/usr/bin/perl

use strict;
use warnings;
use threads;
use WWW::Mechanize;
no warnings 'uninitialized';

open(INPUT,'<','urls.txt') || die("Couldn't open the file in read mode\n");

print "Starting main program\n";

my @threads;

while(my $url = <INPUT>)
{
    chomp $url;
    my $t = threads->new(\&sub1,$url);
    push(@threads,$t);
}

foreach (@threads) {
    $_->join;
}

print "End of main program\n";

sub sub1 {
    my $site = shift;
    sleep 1;
    my $mech = WWW::Mechanize->new();
    $mech->agent_alias('Windows IE 6');

    # trap any error which occurs while sending an HTTP HEAD request to the site
    eval{$mech->head($site);};
    if($@)
    {
        print "Error connecting to: ".$site."\n";
    }

    my $response = $mech->response();

    print $site." => ".$response->header('Server'),"\n";
}

Question:

How to make this program work reliably, and what are the reasons for sporadic crashes?

What is the purpose of calling the join method of the thread object?

According to the document linked below, it will wait for the thread execution to complete Do I call the join method correctly?

http://perldoc.perl.org/threads.html

Please let me know if you have good programming practices that must be included in the above code

Do you need to call sleep () alone or not in your code?

In C, after calling CreateThread (), call Sleep () to start executing threads.

About crashes: when the above Perl code crashes unexpectedly and occasionally, I receive an error message: "Perl command line interpreter has stopped working"

Details of the crash:

Fault Module Name:  ntdll.dll
Exception Code: c0000008

The above exception code corresponds to: status_ INVALID_ HANDLE

Perhaps this corresponds to an invalid handle to the thread

My Perl installation details:

Summary of my perl5 (revision 5 version 14 subversion 2) configuration:

Platform:
osname=MSWin32,osvers=5.2,archname=MSWin32-x86-multi-thread
useithreads=define

Operating system details: Win 7 ultimate, 64 bit operating system

I hope this information is enough to find the root cause of the problem and correct the code

Solution

Your code is correct Maybe your expectations are a little too high

Perl threads are implemented by creating multiple interpreter instances in the same operating system process This isolates Perl code in each thread from all other threads (which is not shared) What it does not (or cannot) do is isolate code that is not under Perl control That is, any module with components written in C For example, a quick look at www:: mechanize shows that if zlib is installed, zlib can be used for compression If this is used and C code is not thread safe enough, it may be a crash problem Therefore, if you want to ensure that your Perl application works well under threads, you must traverse all modules used (and all modules used) and check whether they have non Perl parts or whether they are thread safe For most very important programs, this is an unreasonable workload (or unreasonable restrictions on the cpan modules you can use)

This may be a large part of the reason why threads are not used in Perl