Hello everybody!
I need to mirror sites that need suscription.
This validation is done standardly by the web server.
In this server there is a robots.txt.
I,m trying but I only have succes with public pages.
I need to do it by command line.
Could you tell me how to do a script that worked?
Thanks in advanced! and waiting desperately for an answer
Alicia