How can I circumvent idiot web page designers?

[ChrisGreaves]

A link to the University of Toronto's faculty of Music web site:-
https://music.utoronto.ca/concerts-events.php?cDate=2018-03-29
The link is primed for a week from now.

This bugs the heck out of me:-
I can see that there are three events today (22nd) and only one on other days, somedays two events etc.
Walter Hall is just 15 minutes walk from my home, and these students of performance are all pretty good.
I would like to plan my weeks ahead by finding out what concerts are on at Walter Hall, as distinct from other locations, the dates and times.

The web page merely tells me, fo a single day, that there are "three concerts today", but it does NOT tell me the place, and it does NOT tell me the time.
To find the place and time I am forced to click on the link, make a note or save each individual page, then click to go back.

Why do web designers think that I have nothing better to do than click like a pigeon pecking at a lever?

I'd love to have an application that can d/l the web page and every subsidiary data that can be obtained by clicking on each individual's link.
At least then I could dash of a quick VBA to parse the data and find the stuff I need.

Failing that, I'd appreciate a clue as to how to interrogate the top web page (link given above) so that I can automate the process of downloading data.

For an example (IMNSHO) of a classic web page, see WholeNote. This page lets me see at a glance what days/times/places I should be.
(That I can parse it and drop items directly into my calendar is what I like about being able to obtain all the data).

To my mind, people get confused about "data" and "information". A web page should deliver data to me; it should NOT deliver information that has been processed by the idiot web designer.

I am saying idiot, because the guy actually barged in on my enquiry with the librarian, and petulantly asked me if I had any idea how difficult it was to design a web page. I bit my tongue, thought of you guys, and decided not to tell him that I had designed and written my own web compiler after I'd seen how bad SaveAsHtml was in MSWord. After he left in a snit the librarian apologized to me. I still love librarians.

(signed) "lover of student musicians", of Toronto.

[John Gray]

I also feel your pain.

I cannot count the number of times the monthly newsletter of a well-known cathedral indicates that some person or choir is giving a concert at a certain date and time but doesn't say what the concert is about, let alone the list of the pieces to be performed!
Mentioning this glaring omission to the newsletter editor brought the comment they were often not given the information most normal people would have wanted, so couldn't add it into the newsletter.

I would complain to the appropriate manager of the incompetent web designer, and continue until Something Was Done About It...

[Jay Freedman]

Follow these links (see what I did there? )

https://www.techrepublic.com/article/do ... wnthemall/

https://superuser.com/questions/274414/ ... d-from-one

There are many similar answers; I found these with this search.

[ChrisGreaves]

I would complain to the appropriate manager of the incompetent web designer, and continue until Something Was Done About It...

Been there, done that. The dean of the school sent me an email saying soemthing along the lines of "look into it", but it sounded a lot like "Run along now, Sonny; we have important things to do here".

(signed) "glum" of Toronto

[ChrisGreaves]

https://www.techrepublic.com/article/do ... wnthemall/

Thanks Jay.

This first one failed with a "408 error" on the UofT FoM web site. I will try it on another site.

https://superuser.com/questions/274414/ ... d-from-one

And this one seems to require a utiliuty "wget".
I pasted the first command line as-is and my DOS window told me
"'wget' is not recognized as an internal or external command, operable program or batch file."

The conversation I had with what turned out to be the web page guy suggested that he had slaved long and hard to make this web site incredibly clever.
I suspect he is too clever to write a simple web page with links.

I'll keep trying.
Thanks again.
Chris

[Jay Freedman]

https://www.gnu.org/software/wget/

GNU Wget is a free software package for retrieving files using HTTP, HTTPS, FTP and FTPS the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.
...
Runs on most UNIX-like operating systems as well as Microsoft Windows

Download the Windows version from https://eternallybored.org/misc/wget/

By the way, be sure to read the response by Pablo Halpern near the end of the thread on superuser.com. It points out a number of useful switches in the wget command line.

[John Gray]

Yes, wget is a well-known utility - I use it (v1.17.1) in a BATch file every time I log on to obtain the (current) external IP address of my router from one of the WhatIsMyIP-type websites.

[ChrisGreaves]

Download the Windows version from https://eternallybored.org/misc/wget/

Hi Jay and John.
I must be doing something wrong. I'm probably tired or poorly fed.

WGget.png

I have downloaded four versions (including the 1.17.1 mentioned by John) and all four deliver the same result, an error in Copying.

I am d/l the 64-bit version because I am running the Win7HP 64-operating system.
The zip files are d/l into my B: drive, which is a SUBST drive, but I am extracting them from the path T:\Blotter\20180403\, which FWIW is a TrueCrypt drive.
I get the same errors when I make a folder C:\20180403\ and copy the zip files there before "Extract All".

If you can spot a simple "doh" mistake I'd be grateful; otherwise I think I'll try again tomorrow after I've eaten a couple of meals and slept overnight!

Cheers
Chris

[Jay Freedman]

I don't see any "doh" but I didn't run into any trouble with either the download or the zip extraction. I've temporarily posted the contents of the zip file (for 1.17.1, 64-bit) on my own website for your convenience. Use these links:
https://jay-freedman.info/wget/wget.exe
https://jay-freedman.info/wget/wget.html (documentation)
https://jay-freedman.info/wget/COPYING.txt (GNU license)
https://jay-freedman.info/wget/wget.ini.sample.txt (sample ini file, commented)
https://jay-freedman.info/wget/wget.debug.exe (debug version of the exe, probably not useful to you)

(Note: The last three files are slightly renamed from what's in the zip file, to make them accessible without having to modify the MIME file on the website.)

[ChrisGreaves]

I don't see any "doh" but I didn't run into any trouble with either the download ...

Jay, thank you for your efforts!
I have now d/l the files from your site links.
(True Confession: I was so excited to d/l the exe that I immediately double-clicked on it and was terribly disappointed that it just went "FLASH" and disappeared. Then I slowed down my 71year old CPU and considered running it from a dos prompt ... )

I have now managed to d/l a page or several in PHP format, so I will go home tonight and read the html documentation.
The switches appear to outdo those for RoboCopy.
This could be a good thing or a bad thing (please see "71year old CP" above).

Again, many thanks.

Chris

[Jay Freedman]

You're certainly welcome! I hope you're now able to compile the concert schedule you want.

I'm about to remove the files from my website, because I'm not an official distributor and because that isn't the most current version.

[ChrisGreaves]

Well, this is promising. Here is a snapshot of an excerpt from the d/l Glen Gould School of Music.
Absolutely wonderful!
Now to tackle the dreaded faculty of Music of UofT.
Thanks again to all.
Chris