Now every time I want to update the contents of some file I am serving I need to also update my web content. This would work, but also be frustrating. Would managing this actually be easier than just serving the file over https?

Chances are, you're updating a version number in the filename anyway. Especially if you're using http to allow for distributed http caches. I dunno, does anybody run those to reduce network use when they have a low bandwidth connection anymore?

See also universal SRI, which is sadly not making progress at this time. It’s possible that’s because embedding hashes in pages doesn’t scale properly in practice (as CSP nonces also seem to reflect).

You might like subresource integrity.

Quiet discarding might not always be possible in a reasonable timeframe. You still have to get all the bytes to sha256 the payload, and so what? Do you expect the user to waste bandwidth and wait the check out? Often bandwidth and time both have costs attached.

It could still download it as normal, but show a warning at the end after it computes the hash and compares. Just like getting around a bad HTTPS cert, the browser could allow you to say "I know what I'm doing".

Edit: To clarify, I mean it could show the warning before the browser does the rename of the file from the temporary download file to the final filename.

Considering EXEs specifically (entirely dismissing tarballs and such packages which can easily go up to several gigabytes), you cannot rely on everybody being able to download files in a reasonable time frame. I'm lucky to reside in a country that allows me to download hundreds of megabytes in double-digit seconds, alas that is not true for everybody else out there.

Have we yet constructed a hashing algorithm that maps the file left to right onto the hash in a way that’s provably correct at milestones other than “entire file only”?

If I could prove that the first 5% of a file is intact given the final hash, and repeat that proof continuously as the download proceeds, then it wouldn’t be wasteful.

IANAC

The chaotic nature of hashing algorithms is desirable here because even a couple 'random' bytes changed in the assembly could yield drastically different computations. I don't think you can hash something without going through the whole blob.

Can we mathematically prove that, given

100% hash: 12344567

10% hash: 7251abed

That the 10% hash is a valid candidate for eventually becoming 12344567, assuming the other 90% checks out?

Not to my knowledge, no. I don't even think that you could do that with the most basic checksum algorithm.

Courtesy of Wikipedia on the matter, at the MD5 page:

    MD5("The quick brown fox jumps over the lazy dog") =
    9e107d9d372bb6826bd81d3542a419d6

    MD5("The quick brown fox jumps over the lazy dog.") =
    e4d909c290d0fb1ca068ffaddf22cbd0

The addition of the . at the end has indeterminable effects on the resulting hash. Unless the protocol is defined such that the blob is hashed in blocks of known size and that's what's being checked against (as opposed to the full-blob hash), I don't think it's solvable.

Someone with more street cred on this matter, please correct me if I'm wrong!

EDIT: What about a "Hey, server! I'm 2MBs in, and I got hash blahblohblablablaaah; am I doing fine?" protocol? #terribleideas

You are correct that there is no such single hash; there can't be without it being longer, because then the parts could be brute forced separately.

However, a second hash of 10% of the file would work fine. You can even truncate it if saving a few bytes is that important (since you will check the full file hash later), although I suspect even a page with a bunch of downloads would be fine with a couple of extra hashes per file. While most APIs don't expose this possibility, most hash algorithms can spit out the full hash of the file up to any intermediary points with very little additional work. Or there could be seperate hashes for different blocks of data like P2P protocols do and store them all in a separate file on the https site.

However, the main issue is that these days it is very easy to just use https, most likely easier than any other solution. Browsers encouraging people to care by showing warnings should be very helpful in increasing the number of sites using https.

Torrent files do that, a file is split into chunks each hashed separately.

That sounds like the Parchive index files that are popular on Usenet.

Yeah, you could use 20 hashes.

/s

Why is that a bad idea? I mean, if you're downloading 1GB, your file is split into 1MB chunks, and each hash is 256 bits, that's only 32KB. I doubt anyone would complain about that.

That is exactly what bittorrent does, files are split into chunks and each one is hashed separately.

Then you'd know that you're downloading the right malware?

This exact mechanism is already used for linking scripts/stylesheets from third party websites.