Mods / WebCartographer

Category: #Utility
Author: Th3Dilli
Side: Server
Created: Feb 26th 2023 at 9:28 AM
Last modified: Sep 19th at 9:46 AM
Downloads: 2818
Follow Unfollow 66

Latest file for v1.19.8:
webcartographer_0.5.8.zip 1-click install


This is a utility to export a web map from a savegame file (.vcdbs).
The final output can be hosted using nginx or apache or any other webserver.

Thanks to Drakker who made the website part of this project. vs-webmap

For help, discussion or suggestions join the Discord Server



Demo: Aurafury Ancient Paths Webmap

 

To create a webmap you will also need the client side color exported found here

WebCartographer color exporter  

 

Further instructions on how to use it and set it up can be found on the repository readme.


Export instructions

1. Add the Webcartographer mod to your server
2. Start the server - it will initialize the config with default values
3. Configure what the mod should do using the generated `Webcartographerconfig.json`
- by default it will export to the data path of your vs server or client into the "webmap" folder
- if you want to export when the server starts change `"ExportOnStart": true`
4. If you use a Mode other then 4 you need to add the https://mods.vintagestory.at/wcce to your client and run `.exportcolors` to start a export, this will send the block colors needed to construct the map to the server and start a export. If you use Mode 4 you can still use this to start a export. Once the server has the colors or in Mode 4 you can also use ExportOnStart to export when the server start. You only need to run `.exportcolors` again when changing or updating mods/game to ensure they have the correct color.
5. Wait till it is done, see server console, Note it will by default lock the server and kick players since the server is unusable while exporting
6. In the output directory you will find a html folder that is ready to be hosted using any webserver (nginx, apache, ...) or just open the index.html file in your webbrowser to see the map (markers wont work using that method)

Version For Game version Downloads Release date Changelog Download 1-click mod install*
v0.5.8 47 Sep 19th at 9:46 AM Show webcartographer_0.5.8.zip Install now
v0.5.7 117 Aug 13th at 7:09 PM Show webcartographer_0.5.7.zip Install now
v0.5.6 102 Jul 20th at 10:10 AM Show webcartographer_0.5.6.zip Install now
v0.5.5 107 May 30th at 4:36 PM Show webcartographer_0.5.5.zip Install now
v0.5.4 173 Apr 1st at 7:11 PM Show webcartographer_0.5.4.zip Install now
v0.5.3 80 Mar 31st at 8:30 PM Show webcartographer_0.5.3.zip Install now
v0.5.2 110 Mar 27th at 7:56 PM Show webcartographer_0.5.2.zip Install now
v0.5.1 168 Feb 27th at 11:58 PM Show webcartographer_0.5.1.zip Install now
v0.5.0 68 Feb 27th at 8:31 PM Show webcartographer_0.5.0.zip Install now
v0.5.0-rc.2 159 Jan 27th at 5:15 PM Show webcartographer_0.5.0-rc.2.zip Install now
v0.5.1-rc.1
98 Jan 26th at 10:13 PM Show WebCartographer_0.5.0-rc.1.zip
v0.4.0
160 Dec 28th 2023 at 2:25 PM Show WebCartographer_0.4.0.zip
v0.3.0
216 Oct 26th 2023 at 7:16 PM Show WebCartographer_0.3.0.zip
v0.2.4
205 Sep 6th 2023 at 4:58 PM Show WebCartographer_0.2.4.zip
v0.2.3
173 Jul 14th 2023 at 9:40 AM Show WebCartographer_0.2.3.zip
v0.2.2
231 Apr 30th 2023 at 7:21 AM Show WebCartographer_0.2.2.zip
v0.2.1
151 Apr 29th 2023 at 10:32 AM Show WebCartographer_0.2.1.zip
v0.2.0
216 Mar 8th 2023 at 1:12 PM Show WebCartographer_0.2.0.zip
v0.1.0
226 Feb 26th 2023 at 9:30 AM Show WebCartographer_0.1.0.zip

26 Comments (oldest first | newest first)

💬 Th3DilliAuthor, Aug 13th at 7:09 PM

sorry new version should fix that

 

💬 FrostMannen, Aug 8th at 1:09 PM

I am getting strange SQL errors:

[Server Error] Exception: SQLite Error 1: 'no such column: postion'.
at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
at Microsoft.Data.Sqlite.SqliteCommand.PrepareAndEnumerateStatements(Stopwatch timer)+MoveNext()
at Microsoft.Data.Sqlite.SqliteCommand.GetStatements(Stopwatch timer)+MoveNext()
at Microsoft.Data.Sqlite.SqliteDataReader.NextResult()
at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
at Microsoft.Data.Sqlite.SqliteCommand.ExecuteScalar()
at WebCartographer.SavegameDataLoader.GetServerMapRegionsCount() in /home/dilli/drive_linux/git_repos/vintagestory/mods/WebCartographer/WebCartographer/SavegameDataLoader.cs:line 149
at WebCartographer.Extractor.ExtractStructures() in /home/dilli/drive_linux/git_repos/vintagestory/mods/WebCartographer/WebCartographer/Extractor.cs:line 115
at WebCartographer.Extractor.Run() in /home/dilli/drive_linux/git_repos/vintagestory/mods/WebCartographer/WebCartographer/Extractor.cs:line 99

What can possible make this to happen?

Running VS Server 1.19.8, when trying the older version it all works. But doesn't generate the map as it should :(

💬 Th3DilliAuthor, Jul 20th at 7:45 PM

Thalius

redownload the mod now there was an issue so i replaced the current release file

💬 Thalius, Jul 20th at 3:13 PM

I am trying this mod for the first time, and am not getting a config file generated on my server. Restarted the server a couple of times, and no config file.

 

💬 DejFidOFF, Mar 30th at 7:53 AM

Th3Dilli

webcartographer_0.5.2.zip

"ExportOnStart": true,
"SaveMode": true,
"StopOnDone": false

 

With this options, after finish map exportin server stucks at savemode and people can´t connect to.

💬 DejFidOFF, Mar 27th at 8:31 PM

hooray, thank you for new update !

💬 DejFidOFF, Mar 20th at 8:15 AM

Downside, if you turn the "ExportOnStart": true to false and leave the mod in the /mod directory next server restart it delete all in webmap data folder. It is really bad.

💬 Th3DilliAuthor, Mar 12th at 8:03 PM

DejFidOFF

can you send me a full server-main.txt on discord pls

💬 DejFidOFF, Mar 10th at 10:19 AM

Th3Dilli

No just only one mod in the mod directory 0.5.1

 

💬 Th3DilliAuthor, Mar 8th at 10:37 PM

DejFidOFF

that should mean you have the mod loaded twice (two versions?)

 

💬 DejFidOFF, Feb 28th at 6:03 AM

0.5.1

28.2.2024 06:57:37 [Error] [webcartographer] An exception was thrown when trying to load assembly:
28.2.2024 06:57:37 [Error] [webcartographer] Exception: Assembly with same name is already loaded
at System.Runtime.Loader.AssemblyLoadContext.<LoadFromPath>g____PInvoke|5_0(IntPtr ptrNativeAssemblyBinder, UInt16* ilPath, UInt16* niPath, ObjectHandleOnStack retAssembly)
at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
at System.Reflection.Assembly.LoadFrom(String assemblyFile)
at Vintagestory.Common.ModContainer.<>c__DisplayClass35_0.<LoadAssembly>b__0(String path) in VintagestoryLib\Common\API\ModContainer.cs:line 449
at System.Linq.Enumerable.SelectListIterator`2.MoveNext()
at System.Linq.Enumerable.WhereEnumerableIterator`1.ToList()
at Vintagestory.Common.ModContainer.LoadAssembly(ModCompilationContext compilationContext, ModAssemblyLoader loader) in VintagestoryLib\Common\API\ModContainer.cs:line 448

💬 Th3DilliAuthor, Feb 28th at 12:05 AM

DejFidOFF
try the 0.5.1 this one exports the correct worldextent so if you changed the default spawn that could be the reason for it

 

💬 DejFidOFF, Feb 27th at 11:07 PM

First thank you so much for your work. But, file exported, 125MB in /data folder but index.html output is just only black screen :( ( also sent color data ).

 

💬 Th3DilliAuthor, Feb 23rd at 9:48 AM

Versora

you can hop on my discord and post your logs there, then I can have a look at what might be wrong

 

💬 Versora, Feb 16th at 2:40 AM

I dunno what I be doing wrong but I do not have a webmap folder, and I know I did things correctly.

💬 Th3DilliAuthor, Feb 2nd at 2:30 PM

MKRhere

I do not have telegram but you can join my discord linked above and ask there some stuff if you want

 

the main issue with the standalone approach is gonna be the blockcolors
currently, the game client sets up all the blocks and adds the textures then you can grep a random color from a blocks upwards facing block through the game api
if you do a standalone approach you need to do that manually meaning you need to get a copy of the game client (since that only contains the textures) and assemble all blocks + mod block and extract the up facing texture and get a color
and then extract the color for each block and you have to redo that with every game or mod update

💬 MKRhere, Jan 30th at 10:54 AM

> no one invested the time to make a tool

We're working on a tool to read the save file (and render a map) without launching a server. We're currently going through reversing the binary blobs in the database. We'd love more help from people more involved with the game (modders, etc). If you'd like to help out, even if it's occasionally providing an observation, reach out to me muthukumarthefeathersin, or join our Telegram chat (search for "madrasmc"). Thanks!

 

 

💬 Th3DilliAuthor, Dec 3rd 2023 at 3:32 PM

is there an easier way to do this right now? 
short answere no since no one invested the time to make a tool that can do all at once without having to go through this process

the first issues is the server does not know anything about colors and does not even have the textures
second the game stores the blocks as number id and a table to look them up but everytime you add update a mode that may change, so we can only really rely on the block name but we need to spin up an entire server to resolve the mods , load the data from db and turn that into block data with their names so we can lookup the color that can only be exported on the client


sure this could be made more user friendly and maybe simple but would requiere more work, so unless someone takes up that challange this is the best thing you have right now, the previouse iteration of this "system" was to use a client side mod to export chunk images on the client and then use gdal command line tools to stitch them together to then make a webmap of it so this is already a huge improvement but sure it can be improved even more

afaik their is a mod that is not yet on the moddb, made by VeryGoodDog on github, it is in a usable state but not fully finished as far as I am ware


for the powershell script yeah I am sorry that hasn't been tested in a while, thank you for providing the upated version I will add the to the repo

i have now added instructions of how todo it without using docker to the discription I hope that helps

💬 Drathek, Nov 30th 2023 at 10:43 PM

So this took a bit of effort to get working for a modded server. I'm not sure if the export.ps1 was ever tested because it had syntax errors (one of the if checks was not correct), would break because of existing files (if you ran it again due to lack of -Force usage), and funademental flaws such as Expand-Archive not supporting .tar.gz files.

I haven't code dived the color exporter mod, but it seems very redundant to be running an entire copy of the game to run this mod, have the cartographer mod then iterate over a different save file (to get around the save file being locked), and to rely on the user to preemptively provide a blockColorMapping.json. Can those colors really not be determined server side, and is there no way in an existing game to access the data you need from the save file (despite being able to actually run that save file on the server)?

Additionally, it seems needless to fail if the blockColorMapping has client mods listed; would be better to just make it a warning and continue anyways. If there are actually server mods missing that we need colors for then make them hot pink so the resulting map obviously has problems that require a correct blockColorMapping.json.

If you run the webmap and its broken (console is complaining about ol.js), be sure you aren't blocking cdn.jsdelivr.net.

This is the export.ps1 file I ended up with:

$ErrorActionPreference = 'Stop'
Set-StrictMode -Version Latest

$TYPE = 'stable'
$VERSION = '1.18.15'
$DEST = "$(Get-Location)/cartographer"
$SAVEFILE = "$env:APPDATA/VintagestoryData/Saves/server world.vcdbs" # Change this to whatever your save file is named

docker compose stop cartographer

Copy-Item -Force $SAVEFILE -Destination "default.vcdbs"

if ((Test-Path $DEST)) {
Remove-Item -Path $DEST -Recurse | Out-Null
}
if ((Test-Path map_data)) {
Remove-Item -Path map_data -Recurse | Out-Null
}
New-Item -ItemType Directory -Path $DEST -Force | Out-Null
New-Item -ItemType Directory -Path html -Force | Out-Null
New-Item -ItemType Directory -Path map_data -Force | Out-Null
New-Item -ItemType Directory -Path "$DEST/cartographerData/ModConfig" -Force | Out-Null
New-Item -ItemType Directory -Path "$DEST/cartographerData/Mods" -Force | Out-Null

$SERVER = "vs_server_$VERSION.tar.gz"
$URL_SERVER = "cdn.vintagestory.at/gamefiles/$TYPE/vs_server_linux-x64_$VERSION.tar.gz"

if (-not (Test-Path $SERVER)) {
Invoke-WebRequest -Uri $URL_SERVER -OutFile $SERVER
}
if (-not (Test-Path "$DEST/assets")) {
# Expand-Archive -Path $SERVER -DestinationPath $DEST
tar -xzf $SERVER -C $DEST
}

if ((Test-Path "serverconfig.json")) {
Copy-Item -Force "serverconfig.json" "$DEST/cartographerData/serverconfig.json"
} else {
New-Item -Force -ItemType File "serverconfig.json", "$DEST/cartographerData/serverconfig.json" | Out-Null
}

if ((Test-Path "servermagicnumbers.json")) {
Copy-Item -Force "servermagicnumbers.json" "$DEST/cartographerData/servermagicnumbers.json"
} else {
New-Item -Force -ItemType File "$DEST/cartographerData/servermagicnumbers.json" | Out-Null
}

Copy-Item -Force "WebCartographerConfig.json" "$DEST/cartographerData/ModConfig/WebCartographerConfig.json"
Copy-Item -Force "blockColorMapping.json" "$DEST/cartographerData/ModConfig/blockColorMapping.json"
New-Item -Force -ItemType File "$DEST/cartographerData/Saves/default.vcdbs" | Out-Null
Copy-Item -Force "default.vcdbs" "$DEST/cartographerData/Saves/default.vcdbs"

if ((Test-Path "Mods")) {
Copy-Item -Force -Recurse "Mods" "$DEST/cartographerData"
}

# Doesn't appear to currently be necessary, but it could potentially be
if ((Test-Path "ModConfig")) {
Copy-Item -Force -Recurse "ModConfig" "$DEST/cartographerData"
}

docker compose up cartographer

if ((Test-Path html/data)) {
Remove-Item -Path html/data -Recurse | Out-Null
}
Copy-Item -Force -Recurse "./map_data/world" "./html/data/world/"
Copy-Item -Force -Recurse "./map_data/geojson" "./html/data/"
if ((Test-Path "./map_data/heightmap")) {
Copy-Item -Force -Recurse "./map_data/heightmap" "./html/data/"
}
Remove-Item -Path map_data -Recurse | Out-Null

docker compose up webmap -d

Write-Output "Done creating webmap - Check it out at localhost:4242"

💬 Drathek, Nov 30th 2023 at 10:31 PM

xxpowerofsoundxx The webmap is a lot more useful than simply a large image since you can filter what you see, pan and zoom around, click to get commands to add the waypoint in game, etc; but the exported images you could just tile together to form a map if thats all you want.

💬 xxpowerofsoundxx, Nov 30th 2023 at 2:45 PM

Is there anyway to just export my game map as an image or something instead of all the docker, server, rigamorole? I don't understnad why there's no simple translation too for this. 

💬 Th3DilliAuthor, Mar 6th 2023 at 9:40 PM

apperently ther are some issues running verson 0.1.0 on Windows atm,
to get around those delete WebCartographer.exe.config
and copy everything from Lib64 next to the WebCartographer.exe

💬 tanaKa, Mar 6th 2023 at 6:20 PM

Also a note, I had to copy sqlite3.dll personally into the WC directory.

Config:
{
"Mode": 1,
"DatabaseFile": "C:\Users\admin\AppData\Roaming\VintagestoryData\Saves\default.vcdbs",
"OutputDirectory": "C:\Apache24\htdocs\vcmap",
"ExtractWorldMap": true,
"ExtractStructures": true,
"AbsolutePositions": false,
"ExportHeightmap": true,
"IgnoreSnowBelow": 0,
"ExportSigns": true,
"ExportUntaggedSigns": false,
"TileSize": 256,
"MaxDegreeOfParallelism": -1,
"ExportPgw": true
}

💬 tanaKa, Mar 6th 2023 at 6:19 PM

I can confirm the same error on Windows Server 2019

Looks like it's looking for a personal directory at "C:\Users\Tyron"

 

 

image

💬 Th3DilliAuthor, Mar 3rd 2023 at 7:52 PM

DemonBigj781

could you share your config?
preferable on discord (oh forgot to add the discord link)

💬 DemonBigj781, Mar 3rd 2023 at 12:03 AM

i get this erroe when i set up the server

[Server Error] Failed opening savegame.System.BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
at System.Data.SQLite.UnsafeNativeMethods.sqlite3_config(Int32 op, SQLiteLogCallback func, IntPtr pvUser)
at System.Data.SQLite.SQLite3.StaticIsInitialized()
at System.Data.SQLite.SQLiteLog.Initialize()
at System.Data.SQLite.SQLiteConnection..ctor(String connectionString)
at Vintagestory.API.Common.SQLiteDBConnection.OpenOrCreate(String filename, String& errorMessage, Boolean requireWriteAccess, Boolean corruptionProtection, Boolean doIntegrityCheck) in C:\Users\Tyron\Documents\vintagestory\game\VintagestoryApi\Common\IO\SQLiteDBConnection.cs:line 76

(edit comment delete)