Something is forcing the use of HTTPS

User avatar
Argus
GoldLounger
Posts: 3081
Joined: 24 Jan 2010, 19:07

Something is forcing the use of HTTPS

Post by Argus »

Another odd Fx observation.
Executive summary: trying to use a speed test site; "suddenly" it doesn’t work using Firefox, since the browser (or "something") blocks some content, and the site’s tests are built to use WebSocket and HTTP protocols (in the Advanced tests, if one can get the site loaded, one can select which protocol to use, WebSocket, HTTP or a combination etc.).

(Version 92, but I think it happened with 90-91 as well.)

As we know many sites now only offer pages using HTTPS, but not all. As we also know web browsers have, over time, moved to force the use of HTTPS. And during this time, and still, there have also been some add-ons to help force the use of HTTPS.

All fine, except when you can't use a site because something is forcing the browser to use HTTPS, when the site has something that can't use HTTPS. Here we have, since a long time, an internet speed test site that is using HTTP (run by the folks responsible for our top domain, .se, so should be good quality).

In its quick standard test it use a combination of the WebSocket protocol (less overhead in the communication, full duplex, and no caching, over TCP, which can be good when measuring speed, I guess) and http. In the advanced tests one can pick a couple of cities in SE, and also change these protocols etc.

Some weeks ago, or so, I noticed that I got a blank page with only text, as if you were running without JavaScript on some sites. After that I noticed in the address bar(?) https, that shouldn't be there.

I tried some different things; I disabled an add-on (HTTPS Everywhere; has been working long before this); checked some settings (Only HTTPS was and is disabled in Firefox). Restarted the browser and the site still didn't work, some day later it did ...

But today it's back to its bad habits.
20210918_A.png
Just checked with Fx in "safe mode", i.e. no plugins/add-ons (thus, in my case, since I'm using NoScript, full JavaScript (Fx once moved that setting to config). But no, it still says https://www. etc. and shows a dysfunctional test page. Obvious things checked or done: cleaning cache between tests, and yes the bookmark is "http://www ...".

What is forcing this, the use of HTTPS? Must be something on the page that has changed or Fx's blocking of mixed content has been upgraded.

Things changed during recent years on my side is browser and add-on versions, what's going on at the other end and in between I have no knowledge about or control.

As I mentioned: it has been working before, with add-ons, and now it doesn't matter what; and I tried to remember (hah, good luck without searching) when Fx introduced blocking of mixed (active or passive) content. Then I thought about that, ah, that's not something we use often, I think: with a click on the padlock we get the site information panel, and there's an arrow there. See picture above.

I can temporarily disable protection from mixed content. Then the site works. A click again on the padlock (icon) and we are supposed to be able to enable the protection again, but now there is no button; ah how fun, where did that setting go? :grin:

It turns out Fx started blocking mixed active content in v. 23.
Mixed content blocking in Firefox

Restart the browser and it still works. Clean the cache an sessions with CCleaner, and it doesn't.

Some more checks: ah, Fx developer tools, the console tells me it has blocked this and that.

Solution for now: a temp. disable of blocking mixed content; since the setting will be gone the next time, due to having cleaned the cache etc.

At this point of my still offline post, I tried the whole shebang once more. Clean cache, sessions (and cookies if any), and wait until connections are closed. It worked. Then I repeated what I just did, and it doesn't work. Then some minutes later, and it works. :clown: :groan:

Life was so much easier when: a) software didn't call the different motherships until you had told 'em, now just about anything can invoke that ("Oh, user XYZ123 with IP address ... just opened CCleaner, Excel, Notepad, yadda yadda."); b) software didn't try to be extra supportive.
You do not have the required permissions to view the files attached to this post.
Byelingual    When you speak two languages but start losing vocabulary in both of them.

User avatar
BobH
UraniumLounger
Posts: 9218
Joined: 13 Feb 2010, 01:27
Location: Deep in the Heart of Texas

Re: Something is forcing the use of HTTPS

Post by BobH »

Hello Argus,

In Firefox Tools > Settings > Privacy & Security scroll to the bottom of the page. You will find some options relating to HTTPS.
https.PNG
I don't know if these will resolve your problem but thought they might help.
You do not have the required permissions to view the files attached to this post.
Bob's yer Uncle
(1/2)(1+√5)
Intel Core i5, 3570K, 3.40 GHz, 16 GB RAM, ECS Z77 H2-A3 Mobo, Windows 10 >HPE 64-bit, MS Office 2016

User avatar
Argus
GoldLounger
Posts: 3081
Joined: 24 Jan 2010, 19:07

Re: Something is forcing the use of HTTPS

Post by Argus »

BobH wrote:
18 Sep 2021, 19:42
In Firefox Tools > Settings > Privacy & Security scroll to the bottom of the page. You will find some options relating to HTTPS.https.PNG
Thank you, Bob. I have never used this setting, i.e. it is still in, or was, see tests in posts that will follow below, "Don't enable"; I had an add-on, HTTPS Everywhere, long before Mozilla introduced HTTPS-only Mode in Firefox.

I did know a (tiny) bit about this setting, and I think I mentioned it in one of your threads, and my guess was that you or who it was had it enabled, and for some reason it resulted in an Fx info page about security & cookies.

(Then it's a different matter that HTTP sites are so rare nowadays it's a bit difficult to check how these settings and add-ons work. What we often see, if not full HTTPS, is a mix, some parts traveling over HTTP, and then it gets complicated if it’s active or passive content, when using Fx.)

As I mentioned above, I disabled the add-on, and also checked the setting you mentioned. I am sorry for the lengthy post, a bit of public notetaking again, I guess. I haven't yet done any research, nor contacted the folks running the test site.
Byelingual    When you speak two languages but start losing vocabulary in both of them.

User avatar
StuartR
Administrator
Posts: 12577
Joined: 16 Jan 2010, 15:49
Location: London, Europe

Re: Something is forcing the use of HTTPS

Post by StuartR »

There are other add-ons that also force the use of https, for example I think the DuckDuckGo extension to Firefox does this
StuartR


User avatar
Argus
GoldLounger
Posts: 3081
Joined: 24 Jan 2010, 19:07

Re: Something is forcing the use of HTTPS

Post by Argus »

One idea I had, since I now had disabled my add-on (HTTPS Everywhere) was to enable Firefox's HTTPS-only mode and then add an exception for the above-mentioned site. But then it says you can't add exceptions to private mode windows ..., and I use that a lot. (For the simple reason of not cluttering the Fx history, which isn't much of a history, as soon as you visit a page old records of the same page are gone.)

I had a vague memory of how they introduced HTTPS-only, then something about private mode. Yep, Fx 91 introduced HTTPS by default in Private Browsing. Now what? Is that my recent problem? Why you can’t have exceptions in private mode. And why by default, if they have a setting for all windows or only private mode?

Most Firefox users know that Mozilla has been trying to enhance security/privacy for quite some time, many versions, but not all (me included) read every blog post, follow their developer forums etc. Most don't even check the release notes. Not much different than users of other software.

Gaaah. What a mess Fx/Mozilla's features are. I briefly read some blog posts yesterday, one linked to another, and I was going to check these today. Yesterday I experimented with HTTPS-only mode, mentioned above, and it is enabled in "all windows mode" for the following tests (though I'm using private mode windows).

I go to the support page about mixed content, mentioned above (because I'm looking for the link to one about Fx91 private mode, which probably was from another post).

It says: For more information about mixed content (active and passive), see this blog post.
20210919_A.PNG
What an irony: they link to a blog post about the introduction in Fx23, and bang.
20210919_B.PNG

And you know what, it's an https: link, but I get :censored: warning about going to a site that isn't all https ... What an irony.

It seems Mozilla's implementation of this HTTPS-only is a bit wonky, and if they started introducing this way back in version 23 (at least mixed content), and then added settings for "all windows" or "only private mode windows", why then add it as a default for the latter in version 91. The last bit would explain why you can't have exceptions in "private mode windows".

As I alluded to in my first post, I understand that not all sites or parts of are using HTTPS, thus I have not tried to force it or block if not. That’s Mozilla’s idea. Although I did use an add-on, we tend to want it on certain sites.

Now, after having set HTTPS-only mode to "all windows", as mentioned above and tested it, then disabled it, i.e. returned to the way it was when I started, if I go to the internet speed test site (in private mode window) it still doesn't load, I can use the padlock setting to disable the "mixed content blocking", and the page loads. NOW, I have a button to enable blocking again, I didn't yesterday. And even more, if enable it again the test site works ... but then the button via padlock and site information panel is gone ... maybe the test result page is a tiny bit different than the first, refresh the page it still gone ... Ctrl-F5 and the page doesn't load fully ...

One contributing factor could be NoScript. Of course I have had scripts enabled for this site for a long time, but there can be scripts over HTTPS and HTTP for the same domain.

I give up, must do some more research. And have other work to do. As I said, Mozilla is fiddling with things, although it is far better to work with security than changing UI.
You do not have the required permissions to view the files attached to this post.
Byelingual    When you speak two languages but start losing vocabulary in both of them.

User avatar
Argus
GoldLounger
Posts: 3081
Joined: 24 Jan 2010, 19:07

Re: Something is forcing the use of HTTPS

Post by Argus »

StuartR wrote:
19 Sep 2021, 15:14
There are other add-ons that also force the use of https, for example I think the DuckDuckGo extension to Firefox does this
Thank you, Stuart. Yes, there probably are, and that's one reason why we should be a bit careful not adding too many just because "more security is better", there can be conflicts.

I have only NoScript & uBlock Origin at the moment.
Byelingual    When you speak two languages but start losing vocabulary in both of them.

User avatar
Argus
GoldLounger
Posts: 3081
Joined: 24 Jan 2010, 19:07

Re: Something is forcing the use of HTTPS

Post by Argus »

Not much action here. :grin: So it continues as public notes/research from a one-man band. :smile:

As we all know, posts can be too short or too long. Too short, and it usually omits some information; too long and it obfuscates.
In my first drawn-out post I mentioned that I had checked Firefox’s HTTPS-only mode settings (not that I needed); I also mentioned that I had tried the speed test page in Safe mode, that is, without add-ons. More about that below.

Sorry, Bob & Stuart, that post was a bit too long. Thanks again for your help.
--
Here comes another one, but this time with a solution/conclusion ...

My problem (with or without the add-on for HTTPS) has been the erratic behaviour of Firefox in Private browsing mode using one site that rely on HTTP. I can visit the site with mixed content and it works, a refresh later and it doesn't, then it works for a bit longer etc.

Mozilla has over time added and tweaked features in Firefox. Since quite some time they have something they call Private Browsing mode, other browsers have something similar. The main difference between Normal and Private Browsing mode was, and still is, that the browser doesn't save "browsing information"; cookies, history or session etc. Most of us know this.

(To complicate things a bit, people can of course block all cookies, and white-list the ones they want, no matter normal or private mode, and in the latter case they would always be for the session, and gone when the browser is closed.)

When private browsing mode arrived they and other had to explain that no, private browsing isn't more secure and it's not making you anonymous on the web. It just doesn't save some "tracks" on the client side, i.e. history, cookies and session information.

Then in 2015 Mozilla added something they called Tracking Protection to Private Browsing.

As they have worked with other features they have then added more stuff to the private browsing mode ... Mozilla is muddling things.

They announced Enhanced Tracking Protection (ETP) by default sometime in June 2019, and it came with Fx 69 in September the same year. For some reason they thought it was good idea to make some difference between Normal & Private browsing when it comes to this protection; global cookies & tracking content is blocked in private browsing if using ETP Standard mode ... if you want it blocked in all windows you can use ETP Strict or Custom settings. There can be some explanations for this, blocking in normal mode may perhaps break something so they leave it to Private in ETP Standard mode (which is the default mode for ETP).

But why bother; yes, private browsing sessions can be long, I should know, but the cookies are supposed to be gone when the sessions is over.

Anyhow, we move on with Mozilla adding stuff to Private browsing mode.

February this year saw Firefox 86 introducing Total Cookie Protection. Another one mixing browsing mode with protection mode. I.e. in Private browsing, or if using ETP Strict mode, Firefox Total Cookie Protection kicks in. Every cookie in separate jars, sort of.

And finally, Fx 91. Oh my, it was me who mentioned the release in August ... :grin: Yes, I did read the release notes. But when something breaks in front of your eyes a couple of weeks later ...

From the release notes:
HTTPS-First Policy: Firefox Private Browsing windows now attempt to make all connections to websites secure, and fall back to insecure connections only when websites do not support it. Learn more
Firefox 91 introduces HTTPS by Default in Private Browsing [August 10, 2021]
https://blog.mozilla.org/security/2021/ ... -browsing/

They said at the time that they "expect that HTTPS by Default will expand beyond Private Windows in the coming months." Oh the horror. As it is we still have this HTTPS-only setting, and I asked above, why if it's default in private mode.

One explanation could be that HTTPS by default in private mode "is not directly applied to the loading of in-page components like images, styles, or scripts in the website you are visiting; it only ensures that the page itself is loaded securely if possible. However, loading a page over HTTPS will, in the majority of cases, also cause those in-page components to load over HTTPS."

Things are back to normal, it seems, if I use ... ta-dah: Normal :fanfare: browsing mode when using the abovementioned site.

I haven't looked any closer to what Fx does with its HTTPS-only; but as some may have seen, over the years some users have suggested that Mozilla should include (the add-on) HTTPS Everywhere.

As you were.
Byelingual    When you speak two languages but start losing vocabulary in both of them.

User avatar
BobH
UraniumLounger
Posts: 9218
Joined: 13 Feb 2010, 01:27
Location: Deep in the Heart of Texas

Re: Something is forcing the use of HTTPS

Post by BobH »

2 Thumbs Up!
Bob's yer Uncle
(1/2)(1+√5)
Intel Core i5, 3570K, 3.40 GHz, 16 GB RAM, ECS Z77 H2-A3 Mobo, Windows 10 >HPE 64-bit, MS Office 2016

User avatar
Argus
GoldLounger
Posts: 3081
Joined: 24 Jan 2010, 19:07

Re: Something is forcing the use of HTTPS

Post by Argus »

So much for that. It didn't work this morning.

It seems linked to Mixed Content blocking, introduced in Fx 23 some eons ago mentioned early above, rather than the "HTTPS-only as a default in Private browsing mode" (in the more recent Fx 91).
Byelingual    When you speak two languages but start losing vocabulary in both of them.