Wikipedia Scraping?

Q & A
#1

Hi,

W O W ! ! ! When I see all the hard work you all have done on V3 I feel impressed. Great job, AEON Team! On TOP of the new app a whole new WEBSITE and forum, too! Not an easy undertaking. :clap: :clap: :clap:

You must all be fairly exhausted. Am i right? Well! Time for a cool beverage of you choice! :grinning: :grinning: :grinning:

Version 2 user here. Downloaded the upgrade. Didn’t see anything that jumped out at me in regards to scraping Wikipedia for dates born, lived, places born, etc. Such that if a well-know person or event was in Wikipedia, lots of the granular data would get automatically added in, rather than having to type it in for each new person or event. Was the feature added to V3?

Outlined here: aeontimeline

Sincerely,

Flin

#2

Hi Flin,
I completely understand the need and desire, but to be honest, I don’t think this is a feature we would be likely to introduce into Aeon Timeline 3 directly. It would involve a large amount of effort that would be relevant to a very small subset of our users. I also don’t think it would actually be the best way to approach the task anyway.

I think that type of task is better handled in a different way, via a tool that could do that kind of data scraping and convert it into a format that is able to be easily imported into Aeon Timeline and other programs.

I am aware of a couple of different efforts in that area, and involved in one at a relatively early stage right now, which may help to address some of that need in the future, but there is nothing at a level where it will be useful in the short term.

Matt

2 Likes
#3

There is web scrapers for Markdown, so there will be some kind of sync/import of .md files/folder in Aeon, it will be possible to use the “Markdownload” scraper or similar tools, save the markdown to file in the folder structure that’s synced… Most likely it only save the text…

Another thing that can be done and that works is to use Zotero, scrape the pages you need, export as CSV from Zotero and import to Aeon…

It is also possible to query internet pages directly from within Excel, and store the result in a table in worksheets…

Just some tips about workarounds…

#4

Hi Matt & Jaran,

Jaran! Thanks for the direction!! :smiley:

Matt, you wrote:

I am aware of a couple of different efforts in that area, and involved in one at a relatively early stage right now, which may help to address some of that need in the future, but there is nothing at a level where it will be useful in the short term.

I’d be grateful if you could post more info on those efforts, or send me information via PM. I’d be glad to provide funds as much as I can to get something like this operational. It is something I really need.

Sincerely,

Flin

#5

Hi Flin,

Are you able to send an email to support@timeline.app with details about what exactly you are looking to get implemented? We can discuss things further there.

#6

Yes. Will do. Thank you!