Grabbing a file via auth HTTPS

February 26, 2005 9:11 AM

Using a PHP, Perl, or cron, how do I grab a xml page from a my TiVo that requires https authorization? I have the username and password.

Here's what the url with the username and password looks like. https://<login>:<pass>@<tivo-ip>/TiVoConnect?Command=QueryContainer&Container=/NowPlaying&Recurse=Yes

I don't have access to wget on my hosted site, and cURL didn't work for me. I keep getting bad authorization errors with it.

posted by riffola to Computers & Internet (16 answers total)

curl likes it better if you pass the username and password with the -u parameter. Try:

curl -O -u login:pass https:///TiVoConnect?Command=QueryContainer&Container=/NowPlaying&Recurse=Yes
posted by Remy at 9:21 AM on February 26, 2005

Response by poster: I did use curl_setopt($ch, CURLOPT_USERPWD, $passwd_string); in PHP where $passwd_string = "tivo:$mak"; and that didn't work.
posted by riffola at 9:37 AM on February 26, 2005

Response by poster: In fact here's the script I am trying to get to work. The original poster had the same errors and I thought I could try and fix it, but I can't seem to be able to do so.
posted by riffola at 9:39 AM on February 26, 2005

Have you tried Perl?

e.g.:

#!/usr/bin/perl
print (system("curl -O -u login:pass https:///TiVoConnect?Command=QueryContainer&Container=/NowPlaying&Recurse=Yes"));
posted by AlexReynolds at 10:06 AM on February 26, 2005

Response by poster: With perl, sorry to say but I'm mostly clueless and using what you wrote above in a .cgi file gives me redirection errors.
posted by riffola at 10:17 AM on February 26, 2005

Did you make the file executable? Did you specify the hostname and username:password?
posted by AlexReynolds at 10:29 AM on February 26, 2005

Response by poster: Yep, I did both, and it still didn't work.
posted by riffola at 10:34 AM on February 26, 2005

The error message says the HTTP authentication is using digest (as opposed to basic) mode. Looking at the man page for curl, it appears you may need to add a --digest option to the command line.
posted by Axaxaxas Mlö at 11:01 AM on February 26, 2005

Oh, you're using curl through PHP, so I think what you want is curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY) (the "ANY" should allow it to handle authorization methods other than digest as well).
posted by Axaxaxas Mlö at 11:17 AM on February 26, 2005

Response by poster: The script I am using does use curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY). I am not sure why things are not playing nice.
posted by riffola at 1:23 PM on February 26, 2005

Starting with 7.10, You have to pass the -k flag to allow curl to connect to a server without a valid certificate, which will always be the case when connecting by IP address.
posted by boaz at 5:22 PM on February 26, 2005

That's curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); in PHP, BTW. Don't know enough Perl to help you there.
posted by boaz at 5:28 PM on February 26, 2005

Actually, I finally followed the link and noticed that whoever did it did set that option. Hmmm.
posted by boaz at 5:33 PM on February 26, 2005

Best answer: Firstly, does the address load in your browser? Secondly, try changing CURLAUTH_ANY to CURLAUTH_DIGEST. Then Curl will know in advance which kind to use.
posted by cillit bang at 6:00 PM on February 26, 2005

Best answer: I don't have access to a Tivo, so I can't test anything but a common problem when curl fails, but everything else seems to work is the user agent. Curl by default doesn't set one at all, which can confuse browser sniffers. Also, you may want to specifically set curl to use only digest authentication, i.e CURLAUTH_DIGEST instead of CURLAUTH_ANY.
posted by boaz at 6:02 PM on February 26, 2005

Response by poster: Thanks! It works, all CURLAUTH_DIGEST fixed it. Thanks a lot!
posted by riffola at 7:18 PM on February 26, 2005