Serialising/Deserialising Structured Memory To Disk, AKA Savin' Stuff 

Gotta respect jdh's determination to do things his way. It's a story as inspirational as his plant-growing robot protagonist.

I watched the same exact video that you were describing a few weeks back. It was quite impressive when he started to manipulate the compiler itself to read/write the data in the manner he wanted.

Yh ,I remember. That was pure madness... But he did an entire program memory dumper which could store any program at any point like an emulator. Also it was not ideal for human read.

jdh is cool

​@@hide-347 That video was a roller coaster of emotions. Awe, frustration, confusion, acceptance,

Why the quotes?

Yes - this code was written for teaching, not for being exactly what you'd use in a production environment.,

Also what I do when loading a data file with an unknown number of objects that can have variable amounts of data, is to create an object add it to a Vector or Map, then get a pointer to the object in the container and then add the data to the object referenced by the pointer. Thus, when the data ends I don't have to add the last object outside of the main main data reading loop. Good when adding an object requires a number of steps.

@Abraham Johnathan you're definitely on the right track; I suspect these routines are safe because there's a minimum of manual pointer manipulation, invalid references construct zeros, etc. Running this through a fuzzer would probably find crashes instantly but not code execution. Input parsing is notoriously difficult to get right. (There's entire industries built around this: sqlite3, protobuf, thrift, flex and yacc, etc. Some are horribly unsafe, some are solid.)

@Abraham Johnathan I was not talking about memory corruption or anything like that. What I meant is that with the code as it is now, if a property or property value contains some equal or double quote sign, what you will read back from the file will not be what you initially wrote. For example a property named a=b with value c and a property named d with a string value e,"f,g",h will be written a=b = c d = "e,"f,g",h" and will be read as property 'a' instead of property 'a=b', and with value 'b = c' and property d with some value (not sure exactly what the value will be, it will contain 'e', but then you get what the reader thinks of as a closing double quote, so it's possible that the reader thinks that 'f,g' or 'f,g,h' is a new property name, would need to double check the code to see what exactly happens).

100% agree but it's a fascinating intellectual exercise. That said, he did point out he's been using this format for 15 years, and while JSON was certainly a thing back then, it wasn't as ubiquitous

@@jgcornell of course! Never said anything is wrong with it!

Assume the vector is empty, then the first thing the user does (and no reason why they wouldn't) is set an item at location 5. I need the vector to contain at least 6 elements.

I'd argue the '!' is more easy to overlook. So it's less readable as a result.

@@ScibbieGames One could also write "not", but in such cases I often tend to just put a space between the ! and its argument to make it more visible: 'if (!file.is_open())' --> 'if (! file.is_open())'

that's unreadable

@@nimi5965 if you can't read an early return and a negated statement then you should probably focus on the fundamentals of programming my guy.

@@anarchoyeasty3908 i know how to read i'm just saying why the unnecessary complication

Well it's not to say that it couldn't be done, however, you want to call the open/close functions as minimal as possible. So to have a function branched base on this flag would be tricky because the iostream variants for reading and writing would have to be defined after this bool variable is known. Have an external standalone read and write function that are independent of each other does mitigate the dependency of having to know that value of boolean flag variable. When you open a file stream for reading versus one for writing they are different stream type objects. And the mechanics of reading info into the stream from disk or memory is different than writing it out to disc, memory, or console....

I've used #include before. The biggest issue is you have to pre-process out the comments and trailing commas. And of course in JSON you can't mix subscript arrays with object notation. I've been tempted to just throw LUA into my project, but giving your program arbitrary code execution just to have a nice config file format seems a bit overkill. :P There's one very good reason to use a JSON library in a game engine though, and that's glTF files.

Not necessarily especially when it comes to 3D Graphics, Game Engines... Yes it is quite a good amount of work up front however rolling your own can provide several benefits to you. You have full customization of your own tools that are centered around your own user defined objects. You can customize it by adding new features at will helping to make it easier to allow the user(s) of your product the ability to update to a newer version with ease with less chances of it breaking your application. This also has the benefit that you are not tied to some "standard" or external library or API so that in future cases if they update to a newer version of their library or API and you try to incorporate those changes into your own code base, you don't have to search through your entire source code to find every instance you are using it to make sure all of the changes have been made. When you have a Game Engine that was written / designed by you and or your team as opposed to a 3rd party one, there can be 100s of thousands of lines maybe even a million lines of code you'd have to go through. By writing your own, this decouples the need of relying on that dependency. More than just that, you as the developer of that project has a better understanding of the internal workings and it also allows for more fine tuning. The caveat to it is, there's more work involved and a little more code to maintain. In truth it all depends on the project's current requirements and specifications. This also depends on who you are working for. A given company may require you to use a specific library and or format because they have an existing code base where they have invested millions. A different company may refuse the use of outside libraries due to many reasons such as copyright or the fact that they have their own in house tools. So having the ability to roll you own isn't always a bad idea, but having the ability to do so is just another tool added to your tool box. In the long run it better than not knowing how to do it and just relying on existing libraries. There are pros and cons to both using both approaches. However, there is always an added benefit of knowing when and where to use which approach. Now if you are creating a project from a source of a hobby, then either approach you choose is just as valid as the other. There is nothing wrong with using 3rd party libraries, and there is nothing wrong with writing or designing your own. Also by creating your own file format, it will strengthen your ability to implement file parsers either being in binary or text format. They are great exercises! There is also one more added benefit of rolling your own, creating your own file format is that if you don't release it to the public then it makes it harder for others to "hack" your project because they don't know how your storing your objects in memory. If you use something that is well known like json or xml and they can read it, it's easy for them to manipulate the text file and break it. However, in this video he was using a text file parser as opposed to a binary one. And the intent here is to make the text file human readable. So this is something else to consider. It could also serve as a system of security or anti-cheat by having your own format that hasn't been disclosed or released to the public where it's only available to the team who is working on that project.

Even after reading your edit, sure you rushed to conclusions but I still don't understand would it be a bad idea just because you personally find json similar?

IIRC standard win32 timers are not intended for multimedia purposes where precise timing matters. People use high definition timer functions of WinAPI.

You need to run the timer asynchronously, and then communicate to the other thread when appropriate.

sadly you will need to implement a tree, html works only this way if the html is malformed a new node is not added, if is ok, just add a new children node

Learn about context-free grammars

html is special

Good one! Then also think about how to distinguish the string "javidx" from javidx (with and without the quotes respectively). How about strings containing whitespace at the start/end or within, or that have one or more embedded " characters ;-)

Ah XML: the file format that is simultaneously hard to read as a human and surprisingly hard to parse reliably in code. I for one am glad it's a thing of the past.

@@markmuir7338 Ah XML critics: people that have never really understood the format, but now build tools for other languages that achieve the exact same thing.

@@graealex Yeah, people will criticize XML for being unreadable when it's obvious they've never even tried it. I think it has valid criticisms to be made, but readability isn't one of them.

@@IndellableHatesHandles I think it is actually more readable, because contrary to most formats, the closing tag repeats the name of the opening tag. If you have a complex JSON or YAML, where it is just curly braces or just the indentation level, it gets confusing very fast. Plus I feel like using attributes in a different way than elements also makes it easier to read. Besides that, I think XML makes all the right compromises, people underestimate the abilities and tools surrounding it, yet they claim to not be able to only use a subset of it... and then they rebuild exactly the same tools for a different language, because it turns out, the people that built it for XML didn't do so out of boredom, but because there is a need for it - for example to have schema, or transformation languages. Even the problem of verbosity has already been solved, not once, but a few times (EXI, EBML, WBXML, BML and a few others).

How about yaml or JSON?

I'm not certain but I think string_view might be accessible with C+17 at a minimum however, the C+17 compiler may not provide all of its functionality or features. It's been a while since I've done any programming and C+17 is the latest compiler I have installed on my pc. I don't have C++20 or newer, and I remember being able to experiment with string_view a little bit. So it may be partially supported and I was using the MSVC compiler as opposed to GCC or Clang.

@@skilz8098 Ah you're right, it is a C++17 thing. I guess I was thinking of ranges. 20 is awesome though, you should try it. CppCon has a lot of good talks going over the new stuff. In the past month it's all about 23 though, so you'll have to scroll back a ways.

I mean it's great for learning some basics about serialization and good presentation in regards to that. I didn't mean to come off completely as negative as I sounded :)

If you know what JSON (for example) format is. Then Serializing is converting your objects to this format as a "string", Deserializing is the opposite process. As I get. XD I bet it's harder than this and saving data in various binary formats means it too, but in general, it's something like this. And if it's not, then it's just converting date from your objects, maybe even some other data from variables and this kind of stuff into some ASCII text format.

Java has reflection, so it has no trouble analyzing classes and defining a way to put individual objects and their fields into a binary stream. Besides saving and loading data from disk, this is also useful to pass data over network communication, although for interoperability, many other, often human-readable formats are used. In other applications where bandwidth is at a premium, more compact formats like ASN.1 are used. A lot of stuff in telephony and telematics use that format.

VS has a lot of options, but most can be ignored. You don't have to use every feature, but it's nice to have them there once you do need them. Edit: if you have specific questions, reply here and I'll help you (or share contact info, discord, email, etc.)

@@theRPGmaster Can I contact you on discord or reddit?? I wanted to learn more about cmake integration, source control and in general useful feature to OOP..

@@stefanopilosio6838 You can always search for that online. Documentation helps. Also, you don't need to use Visual Studio if you don't want to. That's up to you. Using a editor with a terminal is a choice that many professionals make.

How often do you need to read one of your files to find why it's not loading correctly? Or create a test file to feed into your program? You can do this with binary files but then you probably need to write a separate translator into a text format. It's usually easier, if slower and less space-efficient, to just use text instead. For game saves, however, you might want some form of obfuscation anyway, just to discourage someone from going in and changing their number of lives to 999999.

Lol sorry, it's there in the PGE repo /utilities, I need to update the link

?

Then you are expecting a life full of read write errors.

@@Yleski probably a joke about jesus quicksaving became he came back 3 days later

JSON External Saving Unbreakable System?

@@TheWeepingCorpse gold lmao