Add Services to Your Laptop Cloud Foundry
Last week I posted a blog describing how to run Cloud Foundry on your own Mac or Linux laptop. If you were interested in the idea, I hope you were able to get it running using the method I described. You might be wanting to run more complex applications, though. Doing that will typically require providing database or messaging services to support your application. In this article, I’ll describe an approach to doing just that – still on your laptop.
Services in support of Cloud Foundry are exposed via an abstract Service Broker API. If you are unfamiliar with this model, I’d suggest reading about Services in this core Cloud Foundry documentation article (link).
In this blog, I’ll use a Cloud Foundry Community component called the “Docker Broker Deployment” to expose commonly used services. That package provides a full Cloud Foundry Service Broker capable of provisioning popular services on-demand. It does this by launching Docker instances as needed corresponding to each service.
Prerequisites
- All prerequisites from my last article (git, etc.)
- Install Cloud Foundry on BOSH lite (as described in the last article)
- Gradle (employed to build the test application) (instructions here)
Let’s get started
On your laptop, you won’t currently have any Service Brokers running, so the “marketplace” command won’t return any results
cf marketplace
The result set is empty — no service brokers.
Compare that with the results you’d see on SAP Cloud Platform’s Trial environment:
cf marketplace
Installing the “Docker Broker” Deployment
The Docker Broker is fairly easy to add to your existing Cloud Foundry installation. I’ll describe a quick installation procedure here. If you are interested in the details and other options available for this deployment, you can consult the README.md file that is part of that community git project.
cd ~/workspace git clone https://github.com/cloudfoundry-community/docker-broker-deployment cd ~/workspace/docker-broker-deployment bosh --deployment docker-broker deploy docker-broker.yml --vars-store tmp/creds.yml \ -o op-cf-integration.yml \ -o services/op-postgresql96.yml \ -o services/op-redis32.yml \ -v cf-api-url=https://api.bosh-lite.com \ -v cf-skip-ssl-validation=true \ -v cf-admin-username=admin \ -v "cf-admin-password=$CF_ADMIN_PASSWORD" \ -v broker-route-name=docker-broker \ -v broker-route-uri=docker-broker.bosh-lite.com \ -o <(./pick-from-cloud-config.sh -o op-cf-integration.yml) bosh --deployment docker-broker run-errand broker-registrar
The final output of the command will look something like this:
You can verify the deployment manually, as well:
bosh –deployment docker-broker instances
And, you can now verify that the Docker Broker is connected to your Cloud Foundry:
cf marketplace
Network Permissions
Even though it isn’t mentioned in the Docker Broker README, I found that I had to explicitly open the ports that will be used by the Docker Broker for service network connections. Here we’ll set up a CF Security Group to open up those ports. On my laptop’s BOSH-lite configuration, these ports are exposed on IP address 10.244.0.144. You will notice that corresponds to the second “docker” instance in the “cf instances” output above – if your IP address list varies from what’s shown above, you may need to do some investigating to determine which IP address corresponds to the exposed service ports.
I’ve placed the required security group configuration in a JSON file that you can download from Github.
wget https://raw.githubusercontent.com/rrainey/cf-local-tools/master/services/docker-broker-sg.json cf create-security-group docker-broker docker-broker-sg.json cf bind-security-group docker-broker mycloud dev
Test an application
Now we have just about everything in place. All that’s needed is to download and push an application suitable to testing a database service. We’ll use one of the Java Spring Boot examples, called Spring Music.
cd ~/workspace git clone https://github.com/cloudfoundry-samples/spring-music cd spring-music ./gradlew clean assemble
cf push
If you look at the project’s manifest.yml file, you will notice a “random-route” directive. That instructs Cloud Foundry to generate a random (and unique) application URL for you – you’ll need to glean that generated URL from the output of the “push” command. In my example run, I can see the URL will be “http://spring-music-scholiastic-ruffian.bosh-lite.com/”. Yours will be different, of course:
When I visit the application URL, I’ll see something like this:
But wait: we haven’t requested a database instance be created yet. What’s going on?
It turns out that Spring Boot will default to employing an in-memory SQL database if you don’t supply one in the manifest. That’s an interesting convenience feature – and you might even decide to use that under some conditions – but it’s not consistent with what we’re trying to do here.
Let’s add a Postgres database instance and connect it to the application.
cf create-service postgresql96 free postgres-db cf bind-service spring-music postgres-db
We must also modify the application’s manifest.yml file to connect to this Postgres instance in place of the in-memory database.
You can modify the file manually using a text editor, or you can simply download the revised file from this Github link. In either case, the file should look like this when you are finished.
--- applications: - name: spring-music memory: 1G random-route: true path: build/libs/spring-music.jar services: - postgres-db env: SPRING_PROFILES_ACTIVE: postgres-cloud
I won’t digress to a discussion on Java Spring Boot, but you can inspect the file spring-music/src/main/java/org/cloudfoundry/samples/music/config/data/AbstractLocalDataSourceConfig.java to locate the tie-in with the “env” directive in the updated manifest.
Run the application on SAP Cloud Platform
Hopefully you are getting the hang of things. We can switch to SAP Cloud Platform’s and run this same application with just a few steps:
- Switch to the appropriate Cloud Platform CF API endpoint using the “cf api” command
- Login via “cf login”
- Create a Postgres service instance
- Run “cf push” and test
Closing thoughts
We haven’t explored it today, but SAP Cloud Platform provides a more enterprise-ready Service Broker implementation. It is called Service Fabrik. Service Fabrik implements a superset of the Open Service Broker API – it adds functionality not only to create and destroy services, but to also to administer those services by providing backup and restore on demand – functions not addressed by the Open Service Broker API today. The Service Fabrik broker is SAP open source – you can explore it at SAP’s public Github group. You can also explore Service Fabrik’s cf command line extensions for backup and restore in this Github project.
Hello Riley,
Thanks for this blog post, I have found it very useful and interesting 😀
I am fairly new working with Cloud Foundry, but I would like to understand, if we are working inside SAP Cloud Platform, is there a way for us as users to add new services to the CF marketplace (living inside SCP) or is this a task restricted only to Service Fabrik team or Cloud administrators?
As part of a little test I am making I have just deployed an small PHP application inside CF but I would like to know if I could (import and) make use of a service such as cleardb to access a MySQL database to complete my example.
Perhaps as a second attempt I should try to create an instance of a HANA service an try to connect there 🙂
Thanks a lot in advance for your response.
Best Regards!
Edgar Martínez.
Hi Edgar,
SAP Cloud Platform does not currently support adding your own Service Brokers, but stay tuned; you are not the first person to ask about that. I was unfamiliar with ClearDB -- Googled it. It certainly looks interesting, but it would almost certainly need to wait for container support in Cloud Platform's Cloud Foundry environment to run it there.
Riley
Hello Riley,
All right, thanks a lot for your response. So, for now, we opted to use postgresql service instead of mysql (to start with). For sure, it would be very interesting to have the option to add our own service brokers and to see new services added to Service Fabrik as well.
In the meantime, thanks again for your blog posts, they were really useful for me and certainly helped me better understand CF.
Best Regards!
Edgar.
Hi Riley,
I've followed your previous blog post on installing Cloud Foundry and this one. Thanks for providing these. I have run into an issue installing the Docker Broker on my Macbook (Running 10.13.3 High Sierra). It seems that the file 'op-cf-integration.yml' is not present. Is this generated in some way?
I've provided the output from the command line below. Thanks for any advice.
Tony
[tjc@mbp ~/workspace/docker-broker-deployment (master)]$ bosh --deployment docker-broker deploy docker-broker.yml --vars-store tmp/creds.yml \
> -o op-cf-integration.yml \
> -o services/op-postgresql96.yml \
> -o services/op-redis32.yml \
> -v cf-api-url=https://api.bosh-lite.com \
> -v cf-skip-ssl-validation=true \
> -v cf-admin-username=admin \
> -v "cf-admin-password=$CF_ADMIN_PASSWORD" \
> -v broker-route-name=docker-broker \
> -v broker-route-uri=docker-broker.bosh-lite.com \
> -o <(./pick-from-cloud-config.sh -o op-cf-integration.yml)
-bash: ./pick-from-cloud-config.sh: No such file or directory
invalid argument for flag `-o, --ops-file' (expected []cmd.OpsFileArg): Reading ops file 'op-cf-integration.yml': Opening file op-cf-integration.yml: open op-cf-integration.yml: no such file or directory
Hi Anthony, I ran into the same issue too. It is because docker-broker-deployment has changed its directory layout, and there are no comments or documentation regarding it.
If you do the following in docker-broker-deployment
and then observe the directory structure, you would notice the differences. Running the command after that works perfectly.