OpenGeofiction

Displaying (route) relations in leaflet multimaps on the wiki?

Posted by Leowezy on 13 August 2017 in English (English).

Hey there,

I have a more or less quick question about, well; I guess the title says it all. I’m not even sure if it’s possible, but I think I’ve seen something like this on the wiki a couple of times, but I might be wrong. The closest thing I actually found was Luciano’s “old” RUTA-VC map, and I wonder whether similar things are possible without an external script. For my personal liking I wouldn’t need to click on the map and have extra info displayed; I’m looking for a way to have a osm-data-fed map that displays route relations for Kojo’s intercity railway services, so I don’t have to bother poor paint (rest in peace) every time I do a tiny change to the network.

Any suggestions welcome! Thanks a lot

leo

Comment from thilo on 13 August 2017 at 19:47

That’s definitely possible. I’m doing something similar with the Relation area calculator, except that in this case the relations are expected to form closed polygons.

Nevertheless, you can type the ID of a railway route into the query field, and the related ways will be displayed (in red, to indicate an error, because no area size can be calculated):

http://tile.opengeofiction.net/util/relation_area.html?q=12548

The script behind it can probably be modified with little effort to display multiple routes at once. I’ll see if I can come up with something.

Comment from Leowezy on 13 August 2017 at 20:15

That sounds great! I wonder whether different routes could be displayed with different colours, and if there is a chance this could then be embedded on wiki articles with (for example) a multimap? I have absolutely no clue about working with this kind of code. Thanks a lot for your help!

Comment from Luciano on 13 August 2017 at 20:36

I was proud of that Ruta/VC multimap, but the whole apparatus of it became unscalable.

And I stopped working on it because I made a decision to redesign VC, so the whole thing has been on hold. But a more scalable solution would be beautiful!

Comment from thilo on 13 August 2017 at 21:52

Here’s a first version:

http://tile.opengeofiction.net/util/route_relations.html?r=12672:FF0000,12681:009900

The “r” Parameter contains of a comma-separated list of relation IDs with hexadecimal 24-bit color values.

Comment from ADB52 on 14 August 2017 at 11:45

How do we obtain relation IDs?

Comment from Leowezy on 14 August 2017 at 17:31

That’s amazing! Thank you so much; very exited to see what I can do with it. I can’t get it to work with my metro relations in Pyingshum right now, but maybe that’s because the ways need to be tagged in a specific way in the route relation?

Comment from thilo on 14 August 2017 at 19:11

@Leowezy: please try again, that should be fixed now.

@ADB52: zoom in so you can activate the “Map Data” layer, then select a way that you know to be a member of the wanted relation. A link to the relation is then displayed at the bottom of the way infobox, like e.g. here:

http://opengeofiction.net/way/4602730#map=19/36.44820/119.24430&layers=D

Comment from Luciano on 14 August 2017 at 23:12

@thilo - that’s pretty cool. Lot’s of potential, definitely.

Here are a few Tárrases bus / light-rail relations:

http://tile.opengeofiction.net/util/route_relations.html?r=44135:666666,44262:009900,44265:FF0000,44266:0000FF

With the Ruta/VC map, I had been tagging route colors directly on the relations (actually, route_master relations), eg

http://opengeofiction.net/relation/12682, where colour=#111111

Is there any way your utility could be driven by route_master rather than by route? And could it retrieve the color from the data?

Comment from thilo on 15 August 2017 at 20:23

@Luciano: here’s version 2. Just omit the color specification from the URL, and the relation will be interpreted as “route_master”:

http://tile.opengeofiction.net/util/route_relations.html?r=12682,12671,12678

Comment from Leowezy on 15 August 2017 at 21:25

Now I’m wondering; I’ve tried to set up a “Master route” for the Papache line D in Pyingshum in relation 76.973, which contains the route relations for the different branches of the D line(s). When I enter the link it zooms to the right location, but nothing is coloured in. Do I need a colour tagging scheme like Luciano, or am I getting something else wrong?

Comment from thilo on 15 August 2017 at 21:44

Yes, the “colour” tag is needed.

Comment from Luciano on 16 August 2017 at 00:16

@thilo - you are a geofiction coding god. This is brilliant.

Check these out…

VC Trams and Light Rail (incomplete) http://tile.opengeofiction.net/util/route_relations.html?r=11939,12700,12801,16827,13591,12839,12869,12854,12836,4547,3644,12671,12744,11910,12657,12739,12704,4760

VC Regional Rail (incomplete) http://tile.opengeofiction.net/util/route_relations.html?r=11779,12766,12756,12549,12759,12765,5839,12706,12708,12682,12678,5841,12652,12662,12459,11998,12654,12664 (might need to rethink the use of the colour WHITE for the Líneas Blancas on these…)

VC Metro (incomplete) http://tile.opengeofiction.net/util/route_relations.html?r=5833,11903,11912,5835,3653,4548,6620,11907,5834,11780,11980,11982

Now, do you like recursion? I made a thing, a long time ago, which I called the route_master_master - relation 5832. Want to try it? It’s just a list of all the member route_masters.

Comment from Luciano on 16 August 2017 at 02:03

Not just trains and buses! You can do ferries, electric grids, and highway routes.

Here are some successful tests for route masters in Tárrases :

Light rail (Línea Gris) http://tile.opengeofiction.net/util/route_relations.html?r=54002

Bus Route 370 (Aérodromo <-> Submarino) http://tile.opengeofiction.net/util/route_relations.html?r=54003

Some ferries http://tile.opengeofiction.net/util/route_relations.html?r=76980,76981

Some electric trunk lines http://tile.opengeofiction.net/util/route_relations.html?r=76982

The main round-island highway http://tile.opengeofiction.net/util/route_relations.html?r=76983

Comment from wangi on 16 August 2017 at 14:52

Those look good; can’t seem to get it working for the Gobras City subway route masters: http://tile.opengeofiction.net/util/route_relations.html?r=66568,66671,66591,66606,66635,66640,66646,66731

Maybe because they are using CSS colour names?

Comment from Luciano on 16 August 2017 at 15:01

@wangi - I’m not the one coding this, but that seems VERY likely to be the cause.

I just retagged one route_master, id=66646, with colour=#ff69b4 [formerly pink]. I also added role “route” to the relation members - I’m not sure if thilo’s code needs this or not. And you’ll have to add the station nodes to the route_master to get them to show with those nifty circles.

That appears to have worked:

http://tile.opengeofiction.net/util/route_relations.html?r=66646

Comment from wangi on 16 August 2017 at 17:12

Thanks, although JOSM now throws a hissy fit because the roles are not empty :)

Comment from ADB52 on 16 August 2017 at 18:21

As a newbe, I’m jumping in at the deep end, following this thread; I’ve added the tramlines in Galmosen to route_master 76998, but can’t seem to make it work - yet.

Comment from wangi on 16 August 2017 at 20:07

ADB52, you’re right to have the relation in your route master relation with an empty role, but the script seems to need them with a role of “route”. Also add a colour tag (British English spelling) to the route master relation.

Comment from ADB52 on 16 August 2017 at 22:30

Shows up the network, but as on a normal map, not with the lines or halts highlighted.

Comment from wangi on 17 August 2017 at 12:08

Yeah, i think the script is only picking stops up from the route_master, rather than the individual route relations.

Your colours need the “#” in front of the values.

The data model on OSM is: the route relations should have the ways, platforms and stops which form the route, in the order of travel. The route master should only have the route relations. See: relation:route_master and relation:route.

Comment from Luciano on 17 August 2017 at 13:51

@wangi - I am aware of the osm standard but as been debated to death in the past, why are we bound to this standard if breaking with it can give us a nice result or simplify our work? Unlike osm, we don’t have people building real-world apps against our data. . . It’s just us. I really don’t understand.

Anyway, a case is to be made for semi-denormalized data structures in any event, to speed queries and simplify coding. To my view, a strong case could be made that the osm public transport schema is badly designed - not to mention it’s a mess in implementation too, full of inconsistencies and half-made decisions.. This further reduces my interest in following it punctilliously.

Comment from wangi on 17 August 2017 at 13:59

Don’t get me wrong - I’m really glad Thilo’s made this script. It’s just that all the documentation out there about relations, route relation and route master relations is for OSM. All the tools are based on that. Deviate from the standard and the validation checks in the tools highlight it.

To me it was clear ADB52 had been reading that documentation, I was helping out with the differences needed to get the thing working for them.

Comment from Luciano on 17 August 2017 at 14:15

@wangi - OK, the issue of documentation is a very strong argument, and actually it’s one I hadn’t quite considered in the way you put it. To the extent we break the standard, we have to take to the time to document it well and make sure other OGF users are aware of it. That’s hard work - certainly I’m not great at it.

One reason why we might want to include the “station/stop” nodes in the route_master in driving this utility is because it isn’t always the case that you want all the detailed nodes from the route relations to show up on the slippy map - you want just one circle/dot per station, where in the route relations, depending on how many pass through a given area in various directions, there might me 2, 3, or even more individual nodes. Thus we use the “display node” (station label) in the route_master, while we use the stop= nodes in the route relations. Maybe thilo could contribute a different perspective - I’m mostly describing the way I solved these problems when I was building for myself, and I don’t know to what extent his solution is being driven by my nonstandard data decisions.

Comment from wangi on 17 August 2017 at 14:27

Yeah, I think station labels in the route master make sense. That then leaves the stop nodes in the routes themselves, on the ways, as expected for routing use? Anyway, the only time I’ve dealt with the stops was on a few of your Tárrases ones above.

Comment from thilo on 17 August 2017 at 21:57

My solution is totally driven by Luciano’s data structure, because I wanted to build a proof of concept, and do it quickly. It wasn’t my intent to provide a general solution.

Nevertheless, my opinion is that we should generally stick to OSM standards whenever possible, even when those are suboptimal. The existing software ecosystem around OSM is being built with those standards in mind. I hadn’t even thought of the documentation, but that’s also an important point.

Now regarding the script, I have to admit that public transport isn’t actually very high on my list of priorities, and I’d be really glad if maybe you guys could take over further development. Please take a look at the page source, it’s only ~ 100 lines of Javascript. If you want to play around with it, you can download the .html file to disk and run it from there, with an URL like e.g. this:

file:///C:/TEMP/route_relations.html?r=54002

Comment from Luciano on 18 August 2017 at 02:40

@thilo - thank you, anyway, for the proof-of-concept.

My javascript skills are quite poor. I did a few attempts during a certain prototyping project 15 years ago. Nothing since.

But this is interesting. I may take a look.

To anyone else reading this, with more competency at coding javascript … it would be awesome to see what could be done.

Comment from Leowezy on 18 August 2017 at 15:53

I had a look at it, and my java skills too are probably not sufficient to really expand on the script at the moment xD So I’m eagerly awaiting what you might come up with, Luciano ;) Otherwise I might even restructure my relations in a way they work with the Java script, rather than the other way round… but anyway, very exciting tool! Thanks a lot

Comment from wangi on 18 August 2017 at 21:04

Thanks for the code Thilo.

Comment from thilo on 18 August 2017 at 21:29

If you have any questions about the code or Javascript in general, don’t hesitate to ask. If you want, I can also augment the code with a few more elaborated comments.

In the optimal case, the end result would be a script that can be integrated into the “Multimaps” Mediawiki extension (which is possible because OGF already runs its own special version thereof.)

Comment from wangi on 18 August 2017 at 21:55

Thanks; can you put the code on the github?

Comment from Luciano on 18 August 2017 at 23:07

@thilo - well, I’m interested enough to look into trying to perfect the code - I agree that integrating to the multimaps would be the way to go. What language is the code base for the multimaps extension? [EDIT: I guess I could have googled that - it’s php, right?] As wangi asked, is it on github (sorry if you said before and I forgot)? Do you have any recommendation for IDE for linux/ubuntu environment? - I’m not happy with my current situation (using gedit for my perl scrips plus my PostgreSQL install).

@wangi - the javascript code thilo made is client side so it’s in the web page html. I was playing with last night but my javascript skill is far too rusty to make any progress.

Comment from wangi on 18 August 2017 at 23:12

Yeah, was looking at the code earlier, was going to make it do the stops off the route relations, plus accept either “” or “route” for the role in the route masters.

Comment from thilo on 18 August 2017 at 23:32

The MultiMaps extension is here:

https://github.com/opengeofiction/mediawiki-extensions-MultiMaps

Because it’s a Mediawiki extension, there has to be some PHP involved, but most of the real action happens in Javascript. The actual map embedded in the Wiki is a Leaflet component, just like on the OSM or OGF main page.

The most relevant files are these:

https://github.com/opengeofiction/mediawiki-extensions-MultiMaps/tree/master/services/OGF

But I wouldn’t even think too much about the MultiMaps integration yet. I’ll gladly take the integration part, once the script as such has reached some degree of maturity.

(Unfortunately I’m not qualified to recommend an IDE for Ubuntu.)

Comment from Subway on 20 August 2017 at 17:04

Hey all,

I fiddled with thilo’s javascript code from above and managed to get a few more things working that might be of benefit. I posted my code on the wiki here. Just take the code and paste it into an html document locally on your computer and run it from your browser.

My changes are: * I removed the entering colours after relations (e.g. 12345:#FF0000) syntax. If the relation is a route_master, it’s taken from the route master’s colour tag; if the relation is just a route, it’s taken from the route relation’s colour tag; otherwise, if the colour tag is missing, it’s default black.

  • It can take a combination of route_master and route relations (for those routes that don’t have a route_master).

  • For route_master relations, stations are taken from the children route relations with role stop, forward:stop, backward:stop, or station. For route relations, stations are taken directly from the same relation with role stop, forward:stop, backward:stop, or station.

  • This means that any stop or station nodes in parent route_master relations won’t show (this shows in thilo’s code).

  • If your route relation (not route_master) has other relations in it (like a stop_area relation), the stations won’t show properly due to how my code is implemented right now.

I haven’t done a lot of testing, things could be broken in some scenarios, as I don’t really know much about the OSM-specifics of the syntax as it relates to Javascript. I’m also not an advanced coder by any means, but I was able to tweak the code to display some of what I wanted better (for my own relations, I didn’t tag stations/stops in the route_master relation).

Thanks, Subway

Comment from Leowezy on 20 August 2017 at 20:25

Thank you subway! I think the way I set up my relalations for the Papachē in Pyingshum, your code would show the station; right now “only” the routes are shown, but it’s already pretty impressive I think. See here

Comment from thilo on 20 August 2017 at 21:15

I did some work preparing the MultiMaps inegration. The result can be viewed here:

http://tile.opengeofiction.net/util/test_01.html

Most of the code for route-drawing has been moved to

http://tile.opengeofiction.net/util/OGFUtil.js

(search for “begin publicTransport”). The code is based on Subway’s version, but note that I replaced the functions from the “Lodash” library (i.e. those starting with “_.”) by their native Javascript equivalents (sorry, my mistake for using them), because I don’t want to package Lodash with the MultiMaps component.

Comment from Luciano on 20 August 2017 at 22:54

@Subway - you’re making more progress than I have, by far. I have no talent for javascript.

I tried it out. I’ve dropped a copy of your script, as-is, on my sandbox server. It works beautifully for route_masters: http://raggedsign.net/ardisphere/Route_relations.html?r=54002

but the output for just routes is quite weird: http://raggedsign.net/ardisphere/Route_relations.html?r=44261

Thoughts?

Comment from Subway on 20 August 2017 at 23:08

@Luciano - yep, your relation 44261 had a weird output because this route relation had child relations within it for the platforms, and my fifth bullet above mentioned that it would break down if there are any relations inside a route relation. I tweaked my code a bit to tell it to ignore such relations (marked with role platform, for example), seems to work on my end. Try it on yours. The code is updated on the wiki page: http://wiki.opengeofiction.net/wiki/index.php/User:Subway/Route_sandbox

Comment from Luciano on 20 August 2017 at 23:49

@Subway - no worries. I took your fifth bullet point to mean simply that stations that were relations wouldn’t show up, not that it would “break” the remaining display. Anyway, I’ll tried your revised code. Works:

http://raggedsign.net/ardisphere/Route_relations.html?r=44261

Comment from Subway on 21 August 2017 at 00:03

@Luciano - yeah, looks good too, sorry I wasn’t clear in my first comment.

I think overall for one line or a small number of lines it looks great. Guess the one small problem remaining I can think of right now (hard to implement) is multiple relations on a single node/way get overlapped on the map and you can only see the top-most colour of the way or node (the colour of the last relation).

Comment from Luciano on 21 August 2017 at 00:10

@Subway - THAT is a hard problem. In fact, that’s where I got stuck with my script-based solution that I was running to generate that old RUTA/VC map that started this conversation. I experimented with map-driven solutions (i.e. creating special parallel ways that were essentially just for display purposes, and enrolling them in relations), but that was unsatisfying. There would have to be a kind of algorithmic solution, involving detecting and offsetting overlaid ways. It’s especially challenging when you want to try to map bus routes, where there’s tons of overlap.

Comment from wangi on 21 August 2017 at 08:19

Set the opacity property on the polylines? Ability to toggle on/off routes?

Comment from thilo on 21 August 2017 at 19:44

MultiMaps integration has now happened, based on Subway’s latest version:

http://wiki.opengeofiction.net/wiki/index.php/User:Thilo/Sandbox

I had to make a few changes to the original source, mostly to replace some functions that won’t be available in the MultiMaps environment, so please everyone use the code in the updated “route_relations.html” as basis for further development.

http://tile.opengeofiction.net/util/route_relations.html?map=12/36.43413/119.33624&r=77465,77576,76904,76973,7690

Comment from Luciano on 22 August 2017 at 01:39

@thilo - thanks for your continuing work on this. Current feedback is that for me your sandbox page “hangs” with “loading map” on the two multimap embeds, and they never load, using chrome (chromium) browser on Ubuntu Linux. They load fine on Opera and Firefox, however. I’ve never had trouble with multimaps on Chrome before - any thoughts?

[UPDATE:] I think I solved the Chrome problem (not sure how! - it was more than just a caching problem), so it’s working fine now. So you can disregard this message. Looks good.

Comment from Leowezy on 22 August 2017 at 14:03

I’m in awe. More than I could have ever wished for. Simply stunning.

Comment from thilo on 22 August 2017 at 21:32

Possible next step from here: figure out a timetable for some routes, and then implement train objects moving along the ways. Proof of concept here:

http://tile.opengeofiction.net/util/route_relations.html?map=13/36.46161/119.54988&r=77576

Comment from Luciano on 23 August 2017 at 00:39

@thilo - I actually was thinking about this. The big question, for me, is, where would that timetable data get stored / maintained? Would it end up in a json object (like we do for the new OGF:Territories map)? As much as I appreciate the efficiency of that, it feels inelegant, as it becomes a kind of ad hoc database.

Comment from Luciano on 23 August 2017 at 05:09

I played around with the multimaps version (from thilo’s sandbox), here:

http://wiki.opengeofiction.net/wiki/index.php/User:Luciano/Sandbox/RouteRelations

Comment from thilo on 23 August 2017 at 20:26

@Luciano - that’s indeed how I’d probably do it. Remember that martinum4 once linked to the GTFS specification in a recent thread:

http://opengeofiction.net/user/ADB52/diary/1516

Those contain a lot of elements that we don’t need, but I’d pick the relevant parts and convert them into some JSON structure, with one file per “route_master” relation, with the relation ID as filename.

That way it would be easy to load timetables for specific routes. I don’t see anything wrong with that. My perspective might well change if the number of timetables got very large and this simple approach became unfeasible, but I wouldn’t want to put much effort into a more elaborated solution unless that’s really necessary.

Comment from Luciano on 23 August 2017 at 23:56

@thilo - I understand what you’re suggesting, with respect to GTFS.

My main point that led to my “inelegant” comment, is that hosting json files seems like it is going to increase admin workload (someone with server sftp permission has to place the files), in a context where we are hoping desperately to decrease admin workload. What kind of solutions exist to automate that process so that it is scalable?

I could brainstorm a possible solution workflow - if a particular user (e.g. myself) is so geeky about public transport that they wish to create fictional GTFS feeds, perhaps it should be that user’s responsibility to find a publicly accessible spot to host their json files, and then we could work out a way to “point” our app (i.e. customized multimaps extension) at those publicly visible feeds. So, for example, I place my GTFS feed on my server (e.g. http://…/transportestarrasenses/gtfs/gtfs.zip) and then I pass that URL to the multimaps functions or enroll that url somewhere, as a one-time event (rather than having to place updated gtfs files every time someone wants to make a change). This seems to reflect duplicating realworld workflows in similar situations.

Comment from thilo on 24 August 2017 at 19:30

What about putting the URL directly into the “route_master” or “route” relation, with a tag like “ogf:timetable_url” ?

The drawback with everyone hosting the timetable on their own, is probably that such a system will over time become prone to linik rot.

Maybe the timetable could instead be embedded into a wiki page, inside a preformatted block, like Subway did with his version of the Javascript code. That way the timetable would be automatically backed up with the rest of the wiki. I guess it would be easy to extract the JSON part, maybe with the help of some additional delimiter lines like e.g. “=== BEGIN TIMETABLE ===” or something similar.

Thilo

Comment from Luciano on 24 August 2017 at 22:07

I actually like the idea of embedding it directly in the wiki - much better than my “host and link” proposal.

Then some kind of “scraper” for the wiki, to get the data.

But if we do it this way, we need to establish some standards / formats - because we’re not following any pre-existing standard or practice.

And… would each user place their own article under some kind of pseudonamespace, or would there be a single giant special wiki article (e.g. “OGF:Route Timetables” or somesuch) that users would update with their individual data?

Comment from wangi on 24 August 2017 at 23:59

I think you’d want to keep it as close to content as possible - the individual line articles / network articles? Would help keeping it up to date.

Comment from thilo on 25 August 2017 at 17:43

If the timetable URL is tagged in the route_master relation, then there’s no need for a unified naming schema. Having one giant wiki article is probably a bad idea. Of course people need to agree on a standard for the timetable content.

I didn’t actually think about scraping the data from the wiki, but rather ad-hoc downloading and extracting whenever a timetable URl is encountered in a relation object. Performance-wise, scraping might be the preferable approach, but that can be added later if it turns out to be necessary. Maybe all pages containing timetables should be put into a common category, so the Mediawiki API can be used to query them.

Another scraping method might be to run an Overpass query on all “route” or “route_master” relation containing the “timetable_url” tag.

Comment from Luciano on 26 August 2017 at 05:46

@thilo - I have posted a first draft of a GTFS-style specification for the Gray Line light rail route in Tárrases, here:

http://wiki.opengeofiction.net/wiki/index.php/Transportes_Tarrasenses/GTFS

Some notes on the data:

I used the OGF node id for the stop ids. The stops.txt table in the feed I made is redundant, because that information could be extracted from the route relation.

I used the OGF relation id for the route ids. This provides some opportunity for “back linking” the data, which will help during testing and anyway guarantees that the route ids are unique.

I’m not quite clear on the degree of detail required in the stop_times.txt table - I placed times for the first and last stop for a single “run” - and assume interpolation combined with the frequencies.txt table would provide the means to discover the remaining data points. If this is not true, let me know.

I’m thinking that in the map, we should have a unified tag “namespace” - something like “gtfs:timetable_url” etc. if we want to store other data there where/when appropriate.

So… at your convenience, let me know if this is anything at all like what you had in mind. No hurries. I just thought that for further progress, it would be helpful to have some “test data,” and the Gray Line in Tárrases seemed like a good candidate.

Happy mapping.

Comment from thilo on 26 August 2017 at 19:26

Are stops.txt, agency.txt, and routes.txt necessary? I’d think this information would be available directly in the relation/node data.

With stop_times.txt I definitely think more detail is needed. At least the duration of the stops must be known, Moreover, I can imagine lots of situations where some parts of a route allow for higher speed than others. (To make even more complicated: two ore more routes might share a single stop, which could cause waiting times.)

What I actually don’t understand is the fact that stop_times.txt specifies fixed “time of day” values instead of just the time difference (in seconds) from the start point.

Comment from Luciano on 26 August 2017 at 21:27

@thilo - thanks for taking a look at it. I wrote a lot, just now (below), in response. Please forgive me, and read it only if you want to. I imagine my comments will anyway be of at least some interest to others following this thread and wondering how to set up geofictional transit data.

(NOTE: For non-thilo readers of the following, I recommend spending some time looking at the GTFS documentation (https://developers.google.com/transit/gtfs/reference/), otherwise you won’t understand about 80% of what I’m talking about.)

I agree that stops.txt and route.txt are, strictly speaking, unnecessary. I’m not sure I agree with respect to agency.txt. Then you mention questions pertaining to stop_times.txt and frequencies.txt. I’ll discuss each of these separately, below.

  • stops.txt As I mentioned, stops.txt is redundant, because it matches (should match) the list of stop nodes in the route relation. I provided it here as a kind of “validation” against the data, and to match the GTFS specification. Instead, we can tag that information with e.g. name=, etc. on the stop nodes themselves. It can be extracted from there. Then there is no need for the stops.txt table, especially if we use the node ids as the stop ids.

  • routes.txt Likewise, routes.txt has the exact same data granularity as the route relation, and the information there could be conveyed in tags on the route relation. As I suggested, I’d recommend that non-OSM-standard tags be given some kind of gtfs prefix, e.g. gtfs:route_short_name, gtfs:route_long_name, gtfs:route_desc, gtfs:route_type. So again, I provided it as a kind of “validation” against the OSM data, and if we agree to leave it out, there’s no problem - we just have to make that part of our standard, and we can use the relation ids as the route ids.

  • agency.txt The agency.txt table is not in the same category as the two others - the information is not replicated with its own specific granularity in the OSM data model. It could be “denormalized” (i.e. repeated across all the route relations or route_master relations), but that might prove difficult to maintain. So I would argue that it IS necessary, but of course only if whatever data presentation model we adopt (i.e. the multimaps widget) NEEDS the information. As currently envisioned, the multimaps widget probably DOESN’T NEED the agency.txt data, but I could imagine a future iteration where people might like to see it presented (i.e. as part of the pop-up info on individual nodes or lines - you can see my experiments with this kind of thing on my old version RUTA-VC multimap, where the pop-ups potentially contained quite a bit of information).

  • stop_times.txt Now with regard to stop_times.txt. I understand what you’re asking for. I actually started out with a more detailed listing, but as I found myself just interpolating the times between the start and end times, I wondered if you wanted to handle that interpolation programmatically - the GTFS documentation implies that that is a “normal” method, and perhaps common. As you say, it seems you would need information about the “wait” or “dwell” time at each stop, and I was puzzled by the lack of this info in the GTFS specification, too. If you want to go with an interpolation method, it would make sense to include a “standard assumed dwell time” somewhere - maybe attached to the trips.txt table or maybe attached to each stop node (e.g. a gtfs:standard_dwell= on the stop node?).

Having said all that, interpolation is probably algorithmically complicated - and I wasn’t sure you wanted to do that. If you’ve got some “canned” algorithms or a function library that already can do it, it’s a great and convenient idea, but I certainly wouldn’t want to tackle solving it from scratch, just for this kind of application.

So in light of that, if we aren’t going to do interpolation, then OF COURSE we will need to provide a much larger level of detail for the stop_times.txt table. I’ll set up a single “trip” with the higher level of detail. I need to spend time looking at the “distance between station” data, and think about speeds, and pay attention to which parts of the route are “grade-separated” vs “in traffic”, etc.

  • frequencies.txt My own question, that wasn’t clear to me in looking at the GTFS documentation, was how one goes from the stop_times.txt instance to the frequencies.txt… as you said, it seems like the stop_times.txt info should be in terms of intervals, rather than absolute times. Otherwise, some kind of interpretation has to take place to be able to apply the data from the frequencies.txt table, e.g. first, calculate the intervals from the “representative sample” in the stop_times.txt table, then apply that interval at the frequency specified.

That seems like another “algorithmically complicated” problem: solving the relation between the stop_times.txt and the frequencies.txt tables.

Regardless, I notice I made a mistake with my sparse stop_times.txt data. I’ll fix that.

So… in the spirit of “keeping things simple” for your multimaps widget, do we throw out the frequencies.txt table? I would consider that completely acceptable, but it puts more of a burden on the person setting up the data, since EVERY SINGLE trip has to be documented. In fact, though, I suspect this is how it’s done in the real world, in most instances.

(As a SIDE NOTE: Most of the transit websites I have visited allow access to extremely detailed vehicle-trip-by-vehicle-trip timetables, even for services that are nominally frequency-based. I suspect from a real-world administrative standpoint, such a level of detail is necessary. The Seoul transit agencies (bus and subway) post down-to-the-second schedules and timetables INCLUDING vehicle id numbers for ALL routes - and users can access that info on the web and on smartphone apps. I was on a bus the other day, and I could take the “bus number” (on a sign above the driver’s seat) and find out my arrival time, down to the second, at my destination stop. Little map-based widgets display the location of EVERY vehicle, by vehicle id number, in the entire system, and they are updated in real time. There are schedules for future times, and there is GPS-based real-world data for the present moment. I suspect the schedules are also being continuously modified in response to real-time traffic information and delay information coming from each vehicle. So users don’t see it - they might just see service frequencies. But it’s all there in giant databases.)

Returning the QUESTION AT HAND: Alternately, we could deviate from the GTFS standard and instead ONLY do frequencies, and create some kind of new table to record the “stop intervals”, e.g. trip_stop_intervals.txt, at a slightly different granularity than the GTFS data. That would probably be much easier for users to then set up nice-looking widgets, but at the cost of losing real-world-equivalent data granularity, and requiring a different programmatic solution.

I know this is a lot of information - I get long-winded. Actually, my first version was rather LONGER! Sorry.

tl;dr: I’m happy to deviate from the GTFS standard in any way you deem easiest! This was just a first draft…

Comment from Luciano on 26 August 2017 at 22:01

I have now updated http://wiki.opengeofiction.net/wiki/index.php/Transportes_Tarrasenses/GTFS

It includes 11 specific “trip instances” for morning northbound weekday service. I made a spreadsheet to do the interpolation, so transit times between stations are all the same - it’s not quite realistic, but it’s testable data, maybe.

Comment from thilo on 28 August 2017 at 14:48

No, I don’t actually think that interpolation is that complicated. I just wanted to point out that it’s unrealistic to assume constant speed over a complete route. (And that the information about staying time at the stops was missing.)

Doing complicated calculations, that’s actually what computers are for (and the fact that you used a spreadsheet is proof of that). IMO the preferred method would be the “trip_stop_intervals.txt” approach. That might be still somewhat unrealistic, because the time needed for a trip might vary over a day due to traffic conditions, but I’m OK with ignoring that. Maybe for every route there could be two or three variants.

The most complicated part would probably be to have some kind of “collision detection” to ensure that multiple trains don’t occupy the same station or travel in the same location at the same time.

Of course the best method would be to have real-world GPS data, because users want information about the actual location of the train they’re waiting for, but as we neither have a real world nor users who depend on that information. (Now that I think about it, not having a real world behind it might be the root cause of many of OGF’s problems …)

BTW, on the route_relations.html page, you can now zoom in to any location, and view the public transport relations of that area by pressing the “reload” button.

Comment from wangi on 21 September 2017 at 10:04

If you’re working with route relations a lot then you’ll find PT Assistant a useful JOSM plugin: https://wiki.openstreetmap.org/wiki/JOSM/Plugins/PT_Assistant


Login to leave a comment