A friend of mine told me about the SAP Cloud Platform Portal Challenge 2018-2019 a couple of months ago (thanks Ourives). I've been looking for an excuse to play with real time public transport data for a while, therefore I decided to make a demo on this subject and join the challenge.
goto.Sydney - Public Transport is an app which main idea is to provide public transport information in real time for Sydney, Australia. The app contains details about routes, stops, timetables, accessibility and real time vehicle positions displayed on a map.
Fiori Launchpad: https://flpnwc-c1413ea3b.dispatcher.ap1.hana.ondemand.com/sites/gotosydney
Standalone app: https://gotosydneyapp-c1413ea3b.dispatcher.ap1.hana.ondemand.com/index.html
Data: GTFS
I started the project studying what was General Transit Feed Specification (GTFS) and how it worked. GTFS is a standard for public transit timetables which enables agencies around the world to publish their data in a standardised and structured format. Google has an incredible website describing all details behind this standard.
The second step was a tricky one: where the hell am I going to get this data from? I would like to use real time data, not only the stop locations and timetables. I was searching for data sets from the public agency in my hometown (Porto Alegre, Brazil). Unfortunately, I couldn't find any real time data from this agency to play with. Ok, time to pick another city. More research, more Google time and voilá! I ended up choosing Sydney, Australia. The NSW Government has a beautiful and helpful site with heaps of information about the public data they publish. They have APIs for static (routes, stops, timetables, etc) and real time (bus position, delays, alerts, etc) information. Well done, NSW!!!
Now that I had access to their data, the fun was about to start. I spent more time reading, testing, changing and playing with GTFS data than developing the app. At least for me, it was a little bit confusing at the beginning. The biggest problem I faced was the data set size. GTFS files tend to be huge (the static one, not the real time)! Storing them wasn't an option. My free MongoDB instance is 500Mb only. Consuming them straight from the API would have a huge impact on performance because there is no way to call only one bus details, for instance. It's all buses or nothing!
I was about to give up when I had an insight. It's "just" a Proof of Concept, not a final product which I would charge thousands of dollars, get millionaire in few weeks and move in to an island somewhere in the Pacific. Therefore, I decided to strip the data set and use the bare minimum. No buses, no trains, ferries only. The ferry data set was much smaller than the others, but still quite large. I deleted and transformed and changed heaps of files and fields to better suite my needs. At the end, I had a small data set ready do be consumed by my Fiori app.
Backend: Nodejs on Heroku
Data structure was ready, it was time to develop my backend. The backend is a nodejs app running on Heroku. The app has APIs for the static and for the real time data. Any request to the real time API in the app (/realtime/) triggers 2 internal request to the NSW Government API to fetch Ferry positions and Trip details.
Bonus feature: every time I commit/push a new version to the git repository, it builds the app and deploy to Heroku automatically. It saves me a lot of time. How I wish we had the same in SCP... I know I could achieve something similar writing heaps of scripts which is exactly my point. Another thing, I could have deployed the app to SCP Cloud Foundry, but we're still waiting SAP to fix some issues in our account (come on SAP, it has been few months now).
Frontend: Fiori app
The Fiori app is an "old" Master/Detail, I don't like the Flexible Column Layout (judge me!). As soon as the app is started, it loads static GTFS data from the backend. Sydney Ferries real time positions are loaded once an user selects a route/trip. The app has 3 levels of Master page: Routes > Trips > Stops. Detail page is always available (actually, it depends on the form factor, but you get it). The stops and ferries positions are displayed on Google Maps at the Detail page. As I would like to have my app running on Fiori Launchpad and as a standalone app, I'm loading Google Maps API from my Component.js.
After selecting Route and Trip direction, an user can click on a stop and see the timetable to the destination. Timetable is displayed in a popup and it auto-scrolls down to the next available trip. Future trips have a blue marker, past trips have none. Also, stops and ferry services have an icon to inform whether they have accessible features or not. I could have used real time data to inform delays and alerts for a given trip, but due to time constrains it is not available (next version, perhaps?).
The calls to the backend are handled by a custom JSON Model I've developed last year and published on NPM (https://www.npmjs.com/package/openui5-model-json-crud). It does not use jQuery AJAX as the standard JSON Model, it uses Fetch API and offers a CRUD interface.
Frontend: Fiori Launchpad on SCP Portal
For the Fiori Launchpad on SCP Portal, I created a custom theme with few changes: background, colours and logo. The Launchpad is public (it doesn't require logging in), has no timeout and has 1 Catalog/Group and 3 Tiles: Ferry, Bus and Train. Bus and Train are dummy tiles, just placeholders with no apps linked to them. My idea was to showcase public transport information from all modals, I mentioned the data set size constrains above.
Frontend: Fiori Launchpad Plugin (shell extension)
Once I've completed the development, I asked few people to test the demo (thanks Nikhil, Pedro and my wife <3). I noted that even with a description on the tile saying it was a dummy one with no app, they still clicked on it (users gonna be users). To address it, I created a Fiori Launchpad Plugin to display a popup with some information about the demo and try to make clear that some tiles will not work. I've also added a button at the header toolbar (between me button and the logo) which opens this blog when clicked on. During this piece of work, debugging my issues, I found and reported a bug to SAP WebIDE team: Fiori Launchpad Plugin does not build a Component-preload.js.
One more thing...
Do you remember I said I would like to run it as a standalone app? Well, the reason behind it was to be able to offer offline capabilities. I've developed it to be a PWA (Progressive Web App) offline enabled. Running the standalone app version, an user can access all routes, trips, stops and timetables when offline. Which I think it is pretty cool, huh? If an user is not offline, she/he would also leverage on the benefits of PWA as the assets (js, css, etc) would be served from the cache, resulting in an initial loading time improvement.
No, there is no real time vehicle position nor maps when offline. I could have used some cache strategy for Google Maps assets, but I'm not sure about Google Maps policy for it and I don't want to go to jail...
I hope you have enjoyed this blog because I had heaps of fun developing it (and some headaches as well). Feel free to leave your feedback.
Thanks 🙂
goto.Sydney - Public Transport is an app which main idea is to provide public transport information in real time for Sydney, Australia. The app contains details about routes, stops, timetables, accessibility and real time vehicle positions displayed on a map.
Fiori Launchpad: https://flpnwc-c1413ea3b.dispatcher.ap1.hana.ondemand.com/sites/gotosydney
Standalone app: https://gotosydneyapp-c1413ea3b.dispatcher.ap1.hana.ondemand.com/index.html
Data: GTFS
I started the project studying what was General Transit Feed Specification (GTFS) and how it worked. GTFS is a standard for public transit timetables which enables agencies around the world to publish their data in a standardised and structured format. Google has an incredible website describing all details behind this standard.
The second step was a tricky one: where the hell am I going to get this data from? I would like to use real time data, not only the stop locations and timetables. I was searching for data sets from the public agency in my hometown (Porto Alegre, Brazil). Unfortunately, I couldn't find any real time data from this agency to play with. Ok, time to pick another city. More research, more Google time and voilá! I ended up choosing Sydney, Australia. The NSW Government has a beautiful and helpful site with heaps of information about the public data they publish. They have APIs for static (routes, stops, timetables, etc) and real time (bus position, delays, alerts, etc) information. Well done, NSW!!!
Now that I had access to their data, the fun was about to start. I spent more time reading, testing, changing and playing with GTFS data than developing the app. At least for me, it was a little bit confusing at the beginning. The biggest problem I faced was the data set size. GTFS files tend to be huge (the static one, not the real time)! Storing them wasn't an option. My free MongoDB instance is 500Mb only. Consuming them straight from the API would have a huge impact on performance because there is no way to call only one bus details, for instance. It's all buses or nothing!
I was about to give up when I had an insight. It's "just" a Proof of Concept, not a final product which I would charge thousands of dollars, get millionaire in few weeks and move in to an island somewhere in the Pacific. Therefore, I decided to strip the data set and use the bare minimum. No buses, no trains, ferries only. The ferry data set was much smaller than the others, but still quite large. I deleted and transformed and changed heaps of files and fields to better suite my needs. At the end, I had a small data set ready do be consumed by my Fiori app.
Backend: Nodejs on Heroku
Data structure was ready, it was time to develop my backend. The backend is a nodejs app running on Heroku. The app has APIs for the static and for the real time data. Any request to the real time API in the app (/realtime/) triggers 2 internal request to the NSW Government API to fetch Ferry positions and Trip details.
Bonus feature: every time I commit/push a new version to the git repository, it builds the app and deploy to Heroku automatically. It saves me a lot of time. How I wish we had the same in SCP... I know I could achieve something similar writing heaps of scripts which is exactly my point. Another thing, I could have deployed the app to SCP Cloud Foundry, but we're still waiting SAP to fix some issues in our account (come on SAP, it has been few months now).
Frontend: Fiori app
The Fiori app is an "old" Master/Detail, I don't like the Flexible Column Layout (judge me!). As soon as the app is started, it loads static GTFS data from the backend. Sydney Ferries real time positions are loaded once an user selects a route/trip. The app has 3 levels of Master page: Routes > Trips > Stops. Detail page is always available (actually, it depends on the form factor, but you get it). The stops and ferries positions are displayed on Google Maps at the Detail page. As I would like to have my app running on Fiori Launchpad and as a standalone app, I'm loading Google Maps API from my Component.js.
After selecting Route and Trip direction, an user can click on a stop and see the timetable to the destination. Timetable is displayed in a popup and it auto-scrolls down to the next available trip. Future trips have a blue marker, past trips have none. Also, stops and ferry services have an icon to inform whether they have accessible features or not. I could have used real time data to inform delays and alerts for a given trip, but due to time constrains it is not available (next version, perhaps?).
The calls to the backend are handled by a custom JSON Model I've developed last year and published on NPM (https://www.npmjs.com/package/openui5-model-json-crud). It does not use jQuery AJAX as the standard JSON Model, it uses Fetch API and offers a CRUD interface.
Figure 1: list of routes + all ferry stops position
Figure 2: list of trips + ferry stops position + real time ferry position for a given route
Figure 3: list of stops + ferry stops position + real time ferry position for a given trip
Figure 4: timetable for a given trip/stop (auto focus on the next trip)
Frontend: Fiori Launchpad on SCP Portal
For the Fiori Launchpad on SCP Portal, I created a custom theme with few changes: background, colours and logo. The Launchpad is public (it doesn't require logging in), has no timeout and has 1 Catalog/Group and 3 Tiles: Ferry, Bus and Train. Bus and Train are dummy tiles, just placeholders with no apps linked to them. My idea was to showcase public transport information from all modals, I mentioned the data set size constrains above.
Figure 5: Fiori Launchpad
Frontend: Fiori Launchpad Plugin (shell extension)
Once I've completed the development, I asked few people to test the demo (thanks Nikhil, Pedro and my wife <3). I noted that even with a description on the tile saying it was a dummy one with no app, they still clicked on it (users gonna be users). To address it, I created a Fiori Launchpad Plugin to display a popup with some information about the demo and try to make clear that some tiles will not work. I've also added a button at the header toolbar (between me button and the logo) which opens this blog when clicked on. During this piece of work, debugging my issues, I found and reported a bug to SAP WebIDE team: Fiori Launchpad Plugin does not build a Component-preload.js.
Figure 6: Fiori Launchpad Plugin (shell extension)
Figure 7: goto.Sydney (flp)
One more thing...
Do you remember I said I would like to run it as a standalone app? Well, the reason behind it was to be able to offer offline capabilities. I've developed it to be a PWA (Progressive Web App) offline enabled. Running the standalone app version, an user can access all routes, trips, stops and timetables when offline. Which I think it is pretty cool, huh? If an user is not offline, she/he would also leverage on the benefits of PWA as the assets (js, css, etc) would be served from the cache, resulting in an initial loading time improvement.
No, there is no real time vehicle position nor maps when offline. I could have used some cache strategy for Google Maps assets, but I'm not sure about Google Maps policy for it and I don't want to go to jail...
Figure 8: goto.Sydney offline mode on! Yayyy \o/
I hope you have enjoyed this blog because I had heaps of fun developing it (and some headaches as well). Feel free to leave your feedback.
Thanks 🙂
- SAP Managed Tags:
4 Comments
