Workflows with Markdown

[Jaran]

@Dormouse

OPML is somewhat limited

Yes, in compare to the .mm format that i.e. Freeplane use as default, the OPML format is somehow limited.

Another limitation of exporting from OPML or .mm to Markdown is that the export or transcoding scripts is mostly limited by the 6 levels of headers that Markdown support, so if you have advanced mind maps with a deeper structure than that, you will not get it correctly “converted/transcoded”.
And, all the exporters and transcoders/converters I have found (open source and free) do not transcribe the mind map to multiple files and folders, it creates the whole mind map as one markdown file with x levels of headers (and no more than 6 levels with headers) the rest is “just” a string of “#”'s because markdown doesn’t support more than the 6 levels.

So say you have a mind map with 15 levels you will have a large job restructuring it into folders and files, and you will need to use a software that do support creating new notes and folders out of selected text, or write a script that does that job…

I have a relatively simple genealogy mind map for one person with 10 levels and 882 “Nodes”, so a relatively small mind map… and it was a nightmare to try to create a useable file and folder structure out of the markdown file exported from Freeplane.

The same problem occurred when trying to use the tools in Scrivener.

It might not be a problem for everyone, specially for small mind maps, but for those that have large mind maps and want them into folders and files, it will be a lot of work getting a mind map file converted to a files and folder structure if you can’t do some kind of scripting/programming.

The only real solution I have found so fare, but not tested yet because I need to clean my Freeplane Project, is to make a Power Query import of a .mm file (or use the import xml in the dev tool), create one or more tables out of that, export to csv and use the csv2md python script found on github, that can create a markdown note for each line in a csv file, and where you can set parameters for what you want to be parent folders etc.
As i wrote, I have not tested this yet…

But if anyone have found some good scripts for when you don’t just want a single md file for your whole mind map, let me know please…
I am thinking about solutions for those that are not scripting or programming gurus…

[Dormouse]

@Jaran

Another limitation of exporting from OPML or .mm to Markdown is that the export or transcoding scripts is mostly limited by the 6 levels of headers that Markdown support

Of course, you are right that a markdown document is limited to 6 levels of headers. Asciidoc ditto. Org-mode many more. But this is a limitation of markdown rather than OPML. There is no reason why a markdown or an OPML export aimed at ending with markdown should be limited to one file. It’s simply the case that any export aimed at markdown would need to be designed to accommodate that limitation. I don’t know if Ulysses has a limitation, but Scrivener’s is smaller than markdown’s.

@Jaran

it was a nightmare to try to create a useable file and folder structure out of the markdown file exported from Freeplane

I’m not really surprised. Many programs claim to export to markdown, but a complex structure requires decisions and the program’s decisions might make little sense to you. In this case, export to CSV or xls ought to make more sense, though they’d still have to make decisions about the exported structure.

I these the decisions are what Matt was referring to here:

@matt

@Dormouse

So, my ideal workflow for that would be that they import/export/synchronise those fields with an originating markdown file.

I guess understanding what format that Markdown file should take would be a challenge.

I could picture it being relatively easy, for instance, to take a Timeline Narrative and turn it into markdown – each step down the tree using progressively smaller headers, with perhaps the item summaries being included as text in between that you could then extend into your writing.

Pulling back out of that structure may be a little more difficult though.

[SCN (Steve)]

@Dormouse

Plottr is very visual and mouse friendly. I can see potential overlaps (eg scenes and character & tag filters). It is set up for series, which Timeline isn’t…

I spilled my coffee. It is the ability of Aeon Timeline 3 to handle a series so powerfully that led me to jump ship from Plottr.

Plottr “seems” to support a series, but each novel is in many ways siloed from the others. Not so with Aeon Timeline, where I can put all the novels into one project without barriers between them.

[Dormouse]

Plottr is very much about separate books making a series. Especially those with essentially standalone plots with some continuity of cast, settings and a few themes. What it supports is the transfer of those elements when a new book is being planned.

Aeon is much better for a saga that splits into volumes.

[SCN (Steve)]

I think Plotter is an excellent tool, and it’s quite popular for a reason. I’ve used it myself. Certainly the ability to transfer elements from one novel to a new novel project is highly useful.

I guess the thing for me is that with AT3, there is no need to create a new project just because I’m writing the next novel in a series. I can just keep using the same project with all of the elements – timeline, characters, objects that I need to keep track of, character arcs, world, etc. – already at my fingertips. I can even write asynchronously, for example, writing the last scene of the last novel before I’ve started writing anything else. As I write, I can look at my story within a novel or the stories across my novels, according to pretty much any criteria I choose to define. I can keep it all coordinated with Scrivener through syncing.

I’m not arguing in any objective sense that one piece of software is better than another, only that, for me, I’m very glad to have found AT3, for what that’s worth. Others feel the same about Plottr or other such software. It’s all good!

[Dormouse]

@SCN

I guess the thing for me is that with AT3, there is no need to create a new project just because I’m writing the next novel in a series.

But isn’t that true for Plottr too?
tbh I don’t use either for plotting. AT3 for managing detailed timelines (when that’s necessary) and Plottr I sometimes use just to have a look at various views on how something is shaping up.

[SCN (Steve)]

Well sure, sort of. When I found out that the so-called novel series features of Plottr only retained certain elements from the prior novel, I tried to use a single novel project for my novel series. As I recall, in a single project it was difficult to define the story beats per each novel and also to plan the beats of the novel series itself. I could “kinda sorta” make it work, but it was as if I was constantly fighting the software and looking for workarounds.

[Dormouse]

Yeah, I think it’s designed for series writers using the same characters and locations but not where there’s a plotting for the series as a whole. More Poirot than Harry Potter.

[SCN (Steve)]

I think you’re right.

[Peter_T]

A fascinating discussion that has escaped me until now. I too once set out to find the Holy Grail of consistent data management for novel projects, so let me add my two cents.

I settled on the yWriter 7 file format because, on the one hand, I had been using yWriter 5 for a long time with great satisfaction, and on the other hand, the then new yw7 XML format stores a lot of useful metadata, such as date/time and duration of scenes. It is easy to understand and uses a straightforward system for IDs. Furthermore, it has an ingenious mechanism for backwards compatible extensions, which lends itself to customization.

First, I created a Python library with classes for reading and writing the file format. Then followed Python scripts for converting to HTML format, which was the bridge to Office word processing. From this came extensions for OpenOffice and LibreOffice. The HTML intermediate steps were eliminated by ODT read and write routines. Then converters for all sorts of data formats were added, including Aeon Timeline 2.

In the meantime I have quite an extensive ecosystem around the yw7 project file:

… and three or four more unfinished novels

Then I replaced yWriter itself with my own program that I can add new features to my liking. The most important converters are now available as plugins for this application. With this, my personal working environment now looks like this:

In the meantime, I have reached the point where I’m not missing anything. Only the Aeon Timeline 2 converter I want to rebuild; at the moment it synchronizes events and scenes via the title; instead I want to give the scene ID to the Aeon events as a property to be able to change the scene titles when revising.

I’ve released all this as open source software, but I imagine my system is so special that I’m the only one using it.

[SCN (Steve)]

Damn, Peter. That’s impressive.

[Peter_T]

Thank you. Actually I want to finish my thriller now, but yesterday I played around with Obsidian, and now I can’t get the idea out of my head to program such a fancy Canvas as a novelyst plugin …

[Faustyna]

@matt

To turn the question around a little the other way… what would be your ideal workflow?

To be able to put in all of the small technical things into Aeon - Characters, Locations, Events, important dates, the relationships between these things, etc, then be able to add details about everything in another program (or Aeon itself), and be able to reference the technical things in a way that I know they are up-to-date.

I actually don’t really want to, or need, to write my entire book in Aeon. But I do want and need for it to able to either sync with, export to, or become a world-bible. Because Aeon is the best with timelines and dates, and custom calendars, meaning any other program I use for my world-bible won’t be enough and I end up going back and forth between it and Aeon.

With things like adding Locations, or Races, or being able to add info to things like Characters, it’s getting there. But I wish there was more. I can have a Relationship like Parent/Child, but I can’t write in any details or important tidbits there. The list of characters in ‘Child’ won’t be in any specific order in the panel, so I can’t be sure they’re there in something like birth order. I can’t make notes to say that this is a step-child, or this one is adopted. I could make more relationship types to cover that sort of thing, but then it gets unwieldy and I have 73 different ‘relationship’ types because everyone needs a specific thing.

And there’s no place to really put in a larger amount of detail. We have Location type data, but beyond ‘Summary’ or adding your own extra text boxes, there isn’t a place to expand on that. And the text boxes we get aren’t great for actually writing in. A pop-out editor like you mention later would be great. Throw in wiki-links there too and that might be all some of us need.

If being able to add details like that in Aeon would be too much, I’d love to export it to something like Obsidian where I can add extra things. Turn it into YAML/Metadata. Obsidian now allows for lists and wikilinks in its metadata. I’ve been working on a way to turn the csv Aeon export into md files with the relationships as YAML, and I just about got there, but Aeon at the moment only really exports half the data. In a Parent/Child relationship, we only get the Parent column filled in. If it’s a bidirectional relationship, like Spouse, only one person gets a spouse, and the other one is just blank.

In an ideal world it would sync in Obsidian as well. But in such a way that it only edited/changed the YAML, letting me add details into the body of the notes. Unsure if that’s possible, but we’re talking ideal here, so let’s go with it.