PDA

View Full Version : Use of curl


benwiggy
11-06-2011, 10:41 AM
I'm trying to download a sequence of jpegs that are on a website.

The url is of the form:
http://domain.com/folder/?PPN=PPN631926003&PHYSID=PHYS_00xx

where xx is from 01-41.

I thought that:
curl -o file#1.jpeg http://domain.com/folder/?PPN=PPN631926003&PHYSID=PHYS_00[01-41]
would work, but it doesn't. (The man pages say to use -o after the URL, but that gets a worse result.)

Is there a problem because of the ?form data?

hayne
11-06-2011, 11:35 AM
I'm guessing that you are having a problem due to the shell (Bash) interpreting some of the characters in your URL - e.g. the '?'.
Try enclosing the URL in single-quotes to protect it from interpretation by the shell.

dmacks
11-06-2011, 12:56 PM
Shell-globs (giving the range of numbers) also sometimes only works based on local filename tests. So you might wind up trying to fetch a URL that actually says PHYSID=PHYS_00[01-41] instead of PHYSID=PHYS_0001 and PHYSID=PHYS_0002 and so on, or else fetching nothing because you do not already have local filenames to match. So in addition to trying hayne's quoting idea, also try just one file instead of a whole bunch to see if the problem is "downloading at all" vs "downloading a bunch". And be sure to tell us the exact thing that does happen rather than just saying that it doesn't do what you would like (easier to trouble-shoot).

acme.mail.order
11-06-2011, 06:36 PM
I thought that:
curl -o file#1.jpeg http://domain.com/folder/?PPN=PPN631926003&PHYSID=PHYS_00[01-41]
would work, but it doesn't.

Any URL with a query string needs to be quoted or the shell grabs the part after the question mark.


curl -o file#1.jpeg 'http://domain.com/folder/?PPN=PPN631926003&PHYSID=PHYS_00[01-41]'

benwiggy
11-07-2011, 03:14 AM
Yes, many thanks.

Always with the quotes.:rolleyes: