[OPEN] 'Error Loading Paths' making games/maps inaccessible

Kirjoittanut Dave, 06.08.2021 at 02:48

Okay, thank you both for the info. I *think* I know what it is now. Unfortunately its not an easy fix (of course it isn't -- this is atWar). It will take some time before I can do a proper fix.

For the moment I put in a "cheap" fix, which *should* make you able to play again. However, the problem with the cheap fix is that it has to be individually applied to every single map, and it would take me ages to do them all. So I only put the cheap fix on the World Map for now.

@geyeluvu could you try to play the World Map and see what happens? If it works, then at least I'll know I'm on the right track...

I got the same thing as @geyeluvu - it had the error message, then said 'reloading paths, please wait' and I was able to access the map.

Kirjoittanut humanist, 06.08.2021 at 09:44

Thank you; it worked! There was an error message, but then it said something like reloading.
I tried to enter another game, but it loaded so fast that I couldn't read the messages, but I can play the world map. Does this fix help the other players as well?

Okay, cool. That'll fix it for every player, but like I said only on the World Map. I'll have to update all the other maps when I do a proper fix.

Kirjoittanut BrnoLeopard, 06.08.2021 at 13:31

Kirjoittanut humanist, 04.08.2021 at 12:58

Kirjoittanut lardochecola, 04.08.2021 at 10:47

I tried with all the games that were available, I tried to start some games with different maps but always the same error. in the console it does not show anything, but here is a screenshot
https://imgur.com/3WfToac

We must be cursed.

I still have the same problem.

Me too, unfortunately...

Dave fixed world map, and it worked for me. Have you tried world map?

Kirjoittanut humanist, 06.08.2021 at 14:30

Kirjoittanut BrnoLeopard, 06.08.2021 at 13:31

Kirjoittanut humanist, 04.08.2021 at 12:58

Kirjoittanut lardochecola, 04.08.2021 at 10:47

I tried with all the games that were available, I tried to start some games with different maps but always the same error. in the console it does not show anything, but here is a screenshot
https://imgur.com/3WfToac

We must be cursed.

I still have the same problem.

Me too, unfortunately...

Dave fixed world map, and it worked for me. Have you tried world map?

Yes, world map works for me as well.

Not to be mean, but do you know when the fix will be done?

Kirjoittanut BrnoLeopard, 16.08.2021 at 19:47

Not to be mean, but do you know when the fix will be done?

Maybe we're the only players with difficulty accessing maps.

Kirjoittanut humanist, 18.08.2021 at 11:59

Kirjoittanut BrnoLeopard, 16.08.2021 at 19:47

Not to be mean, but do you know when the fix will be done?

Maybe we're the only players with difficulty accessing maps.

I guess, I'm not sure...

@Dave You still there?

Kirjoittanut BrnoLeopard, 21.08.2021 at 16:01

@Dave You still there?

I think that we must have patience. Dave will fix the issue.

Kirjoittanut humanist, 26.08.2021 at 08:05

Kirjoittanut BrnoLeopard, 21.08.2021 at 16:01

@Dave You still there?

I think that we must have patience. Dave will fix the issue.

Yes... Patience is key.

Kirjoittanut BrnoLeopard, 16.08.2021 at 19:47

Not to be mean, but do you know when the fix will be done?

I really have no idea. It could be weeks, months, or years. It's a complex problem (I'll explain why in a minute), and of course the time I can dedicate to atWar is somewhat limited.

Kirjoittanut humanist, 18.08.2021 at 11:59

Maybe we're the only players with difficulty accessing maps.

You're the only 2 who have reported this specific issue to me. In programming there's a rule of thumb -- for every 1 user who reports a bug, there are probably 10 others who say nothing about it. So certainly there could be others effected by this issue, but in any case, with approx. 7K active users at the moment, it must be a pretty tiny percentage. I still consider it to be important, but its not super-urgent either, since the vast majority of people are not effected.

(Yes, there are tons of other bugs out there that lead to "error loading paths", but they are caused by unrelated problems, so not relevant to this discussion.)

---

Okay, so why is this a complex issue?

The problem you are experiencing is that, for you, Zip.js is failing. Zip.js is a JavaScript library for compressing/decompressing .zip files. AtWar uses this because the borders (or "paths") of any given map are stored somewhere on the server as a .zip file. When you load a map, a bunch of stuff is loaded by your browser, including the .zip files containing the paths, which are then decompressed by Zip.js.

Why is Zip.js failing for you, but not for everyone else? That's the mystery. For me, Zip.js always works... I've tried multiple devices, multiple locations, and multiple browsers (Chrome, Firefox, and Edge). I've never gotten it to fail on my side. The fact that it doesn't work for you means that something *must* be interfering on your side, but I don't know what it is.

If I were to guess, I'd say most likely you have some browser extension that is blocking or causing a conflict with Zip.js. If you use an incognito tab (Chrome) or private tab (Firefox), that would bypass all your browser extensions and probably be able to load atWar like normal. However if that doesn't work, then it must be something even more unusual. Maybe you have a firewall that blocks certain scripts, and for some reason is blocking Zip.js. Or maybe your internet service provider is interfering in some way. I know some ISPs will do funny things to compress data and therefore reduce bandwidth usage... perhaps they are compressing our (already compressed) .zip files in a way that makes them unreadable by Zip.js. (That's just a wild guess on my part, but it would be interesting if you could try loading atWar from a different location, maybe a coffee shop or something, and see if it makes any difference... just for sake of solving the mystery of why this is happening...)

Anyway, regardless of what's causing Zip.js to fail for you, I've already been thinking for a long time about getting rid of Zip.js. I assume the original developers of atWar used it to save space on the server. However the drawback is that it makes games take longer to load, and uses more CPU resources on the client side (i.e. on your side). It's one more thing that contributes to atWar feeling slow or clunky, especially if you happen to have a computer that's maybe a little older. In my opinion, it should be removed.

What needs to be done to completely remove Zip.js from atWar?

The first problem is that all maps' borders are currently saved as zip files on the server. If we get rid of Zip.js, then the borders need to be saved in a format your browser can understand without having to unzip them. Basically, we need to unzip the borders of every map on the server and use the original uncompressed files instead. Unfortunately, there are currently over 17K maps (17,759 as of right now) on atWar -- far too many to go through manually. So my first task would be to write a script that can automatically go through them all and unzip the stuff. But actually that leads to another problem -- unzipping all those files would exceed the amount of storage space we have available.

We're very tight on available space as it is -- in fact, I'm continually having to move stuff off to other servers to make room. The maps however would be difficult to move because pretty much all of the atWar code expects them to be right there on the main server, where they are now. So that leads me to another task that would need to be done first -- we need to free up some space.

I think a big percentage of our 17K maps can be considered "junk" -- either broken maps, or maps that people cloned and then never did anything with. I know a lot of people have asked for the ability to delete broken maps. So maybe the first thing I'll need to do is add a "delete map" feature. Of course I'll have to do this very carefully to prevent abuse, because there are plenty of people out there who would love to maliciously delete popular maps if they could. I've been thinking about this for a long time -- probably I'll build in a requirement that the map can only be deleted if it has less than 10 plays (or something like that), so the more popular maps can't just disappear overnight.

After that, I could even do a big "purge" of junk maps. Maybe I could write a script to delete all maps where there are less than 10 plays AND the mapmaker who owns it has been inactive for, say, 2 years. That would free up a lot of space! Then we could go back and unzip the maps that remain, and we'd probably have enough space to do it.

A little side note: the "temporary fix" I did for the World Map was to first unzip the World Map's borders on the server, and then I added some code that basically says "if Zip.js failed for any reason, grab the uncompressed borders instead". That was enough to make things work again for you guys... but to do this for other maps, I would have to manually go and unzip each specific map that you wanted to play, one at a time. As you could imagine, that would end up being a lot of manual work, which I'd rather not have to do. I'd rather wait until I can automate it, and do them all at once.

Anyway... back to what I was saying. Lets say we've purged a bunch of maps, we've made space, and we've unzipped all the remaining maps' borders... now we're finally ready to remove Zip.js. This involves a couple of things. First, we need to remove it from the Map Editor -- Zip.js is used to create the .zip files that get uploaded to the server in the first place. We need to make sure that when you save a map in the editor, it saves the borders in their uncompressed format instead. This shouldn't be too difficult, but dealing with the Map Editor code is always a bit of a nightmare so I have to allow more time than I would otherwise expect. After that, we can (finally) remove Zip.js from the game. And then we can finally say, "problem solved".

So to summarize, I guess the biggest problem at the moment is storage space and the fact that we have 17K maps to process. The first thing I'll do is add the "delete maps" feature so we can start making space, so keep your eyes open for that. When you see an announcement about a new "delete map" button, you'll know I'm making some progress on this issue as well.

Dave, thanks for investigating. I tried using incognito but no luck. I'll try your other suggestions later. I used a private window with MS Edge.

Kirjoittanut Dave, 30.08.2021 at 07:27

Kirjoittanut BrnoLeopard, 16.08.2021 at 19:47

Not to be mean, but do you know when the fix will be done?

I really have no idea. It could be weeks, months, or years. It's a complex problem (I'll explain why in a minute), and of course the time I can dedicate to atWar is somewhat limited.

Kirjoittanut humanist, 18.08.2021 at 11:59

Maybe we're the only players with difficulty accessing maps.

You're the only 2 who have reported this specific issue to me. In programming there's a rule of thumb -- for every 1 user who reports a bug, there are probably 10 others who say nothing about it. So certainly there could be others effected by this issue, but in any case, with approx. 7K active users at the moment, it must be a pretty tiny percentage. I still consider it to be important, but its not super-urgent either, since the vast majority of people are not effected.

(Yes, there are tons of other bugs out there that lead to "error loading paths", but they are caused by unrelated problems, so not relevant to this discussion.)

---

Okay, so why is this a complex issue?

The problem you are experiencing is that, for you, JSZip is failing. JSZip is a JavaScript library for compressing/decompressing .zip files. AtWar uses this because the borders (or "paths") of any given map are stored somewhere on the server as a .zip file. When you load a map, a bunch of stuff is loaded by your browser, including the .zip files containing the paths, which are then decompressed by JSZip.

Why is JSZip failing for you, but not for everyone else? That's the mystery. For me, JSZip always works... I've tried multiple devices, multiple locations, and multiple browsers (Chrome, Firefox, and Edge). I've never gotten it to fail on my side. The fact that it doesn't work for you means that something *must* be interfering on your side, but I don't know what it is.

If I were to guess, I'd say most likely you have some browser extension that is blocking or causing a conflict with JSZip. If you use an incognito tab (Chrome) or private tab (Firefox), that would bypass all your browser extensions and probably be able to load atWar like normal. However if that doesn't work, then it must be something even more unusual. Maybe you have a firewall that blocks certain scripts, and for some reason is blocking JSZip. Or maybe your internet service provider is interfering in some way. I know some ISPs will do funny things to compress data and therefore reduce bandwidth usage... perhaps they are compressing our (already compressed) .zip files in a way that makes them unreadable by JSZip. (That's just a wild guess on my part, but it would be interesting if you could try loading atWar from a different location, maybe a coffee shop or something, and see if it makes any difference... just for sake of solving the mystery of why this is happening...)

Anyway, regardless of what's causing JSZip to fail for you, I've already been thinking for a long time about getting rid of JSZip. I assume the original developers of atWar used it to save space on the server. However the drawback is that it makes games take longer to load, and uses more CPU resources on the client side (i.e. on your side). It's one more thing that contributes to atWar feeling slow or clunky, especially if you happen to have a computer that's maybe a little older. In my opinion, it should be removed.

What needs to be done to completely remove JSZip from atWar?

The first problem is that all maps' borders are currently saved as zip files on the server. If we get rid of JSZip, then the borders need to be saved in a format your browser can understand without having to unzip them. Basically, we need to unzip the borders of every map on the server and use the original uncompressed files instead. Unfortunately, there are currently over 17K maps (17,759 as of right now) on atWar -- far too many to go through manually. So my first task would be to write a script that can automatically go through them all and unzip the stuff. But actually that leads to another problem -- unzipping all those files would exceed the amount of storage space we have available.

We're very tight on available space as it is -- in fact, I'm continually having to move stuff off to other servers to make room. The maps however would be difficult to move because pretty much all of the atWar code expects them to be right there on the main server, where they are now. So that leads me to another task that would need to be done first -- we need to free up some space.

I think a big percentage of our 17K maps can be considered "junk" -- either broken maps, or maps that people cloned and then never did anything with. I know a lot of people have asked for the ability to delete broken maps. So maybe the first thing I'll need to do is add a "delete map" feature. Of course I'll have to do this very carefully to prevent abuse, because there are plenty of people out there who would love to maliciously delete popular maps if they could. I've been thinking about this for a long time -- probably I'll build in a requirement that the map can only be deleted if it has less than 10 plays (or something like that), so the more popular maps can't just disappear overnight.

After that, I could even do a big "purge" of junk maps. Maybe I could write a script to delete all maps where there are less than 10 plays AND the mapmaker who owns it has been inactive for, say, 2 years. That would free up a lot of space! Then we could go back and unzip the maps that remain, and we'd probably have enough space to do it.

A little side note: the "temporary fix" I did for the World Map was to first unzip the World Map's borders on the server, and then I added some code that basically says "if JSZip failed for any reason, grab the uncompressed borders instead". That was enough to make things work again for you guys... but to do this for other maps, I would have to manually go and unzip each specific map that you wanted to play, one at a time. As you could imagine, that would end up being a lot of manual work, which I'd rather not have to do. I'd rather wait until I can automate it, and do them all at once.

Anyway... back to what I was saying. Lets say we've purged a bunch of maps, we've made space, and we've unzipped all the remaining maps' borders... now we're finally ready to remove JSZip. This involves a couple of things. First, we need to remove it from the Map Editor -- JSZip is used to create the .zip files that get uploaded to the server in the first place. We need to make sure that when you save a map in the editor, it saves the borders in their uncompressed format instead. This shouldn't be too difficult, but dealing with the Map Editor code is always a bit of a nightmare so I have to allow more time than I would otherwise expect. After that, we can (finally) remove JSZip from the game. And then we can finally say, "problem solved".

So to summarize, I guess the biggest problem at the moment is storage space and the fact that we have 17K maps to process. The first thing I'll do is add the "delete maps" feature so we can start making space, so keep your eyes open for that. When you see an announcement about a new "delete map" button, you'll know I'm making some progress on this issue as well.

I thought I had seen something to do with JSZip on my side... I'll see if I can do something with it.

Https://ibb.co/XjPmJk5
https://ibb.co/b1mHqKF

Any idea what I should do with this?

Kirjoittanut BrnoLeopard, 30.08.2021 at 19:17

I thought I had seen something to do with JSZip on my side... I'll see if I can do something with it.

Oops I misspoke slightly.... we use Zip.js, not JSZip. (Both are JavaScript libraries that do the same thing.... I got them backwards when I wrote this.... not that it changes anything)

Kirjoittanut BrnoLeopard, 30.08.2021 at 22:22

Https://ibb.co/XjPmJk5
https://ibb.co/b1mHqKF

Any idea what I should do with this?

There's nothing to do there. The first screenshot is just showing that Zip.js is loaded, which is expected. The 2nd screenshot shows its failing, as I described to you. The problem is somewhere else.

Hmmm...

Kirjoittanut BrnoLeopard, 31.08.2021 at 22:12

Hmmm...

I can access all maps again. I didn't change anything. Any luck?

Update: with the most recent map editor update I've switched the default to use the raw .xaml files and NOT the .zip files. Zips are still there, but only used as a fallback if the xamls can't be found. This has speeded things up a bit. Next step is still the same as I talked about earlier... I need to unzip all from the existing 18K+ maps on the server and then we can get rid of zip.js completely.

From your perspective nothing should have changed, except for maps loading faster now.