I'm glad to post the result of my work for the Data Geek Challenge.
When I read about Data Geek, I was a little unsure if I could deliver a work like this without having almost no experience with data manipulation and creation of viewers that give meaning to these data.
However I was positively surprised, because with SAP LUMIRA the work has become easy and pleasurable. Also, make interesting discoveries about a mountain of information, only fosters the idea that there are many other facts hidden in this huge amount of data that humans produce everyday.
The subject
It was time to put the brain to work and decide what kind of information I would like to manipulate, and make SAP LUMIRA'S visualizations. This part was very easy.
I've chosen to work with Japanese census data to discover more about immigrants population in this fascinating country, and the reason is because I am passionate about Japan since childhood and also because my Japanese wife would be very helpful with language matters, since my skills on the all sort of kanjis are limited to almost zero. Also my 11 months daughter holds both nationalities, which makes me very proud. :razz:
It is far from a complex work, and may not look insightful for many people, however it was a great way to start demonstrating SAP LUMIRA in a easy way (for me at least).
I wanted to know how many Brazilians live in all Japanese cities or prefectures, how many of them were male or female and their ages. Also I wanted to compare the Brazilian population number with other nationalities to discover the position of them among others. With this data it comes together the population of other 9 nationalities of people that are living in Japan, what allowed me to show some insights for them in the end of this post. Also, if you wish, you can download it to check other ideas.
Background
More than one hundred years ago, many Japanese people came to Brazil to pursue a better life, since we had in Brazil plenty of land and coffee farms and no workforce to work in these lands. Brazilian government in an agreement with the Japanese government allowed many Japanese citizens to immigrate to this side of the planet, which made the quest a great adventure for those with enough courage to cross the world in a ship toward South America. Brazil has the most part of Japanese citizens decedents in the whole world, and in the late eighties they started doing the path back, going to Japan to work in all sort of factories and pursuing what everybody does, a better life. And for most of the walking creatures in this planet, a better life only comes with opportunities and hard work.
If you'd like to know more about this fascinating history, please head to http://en.wikipedia.org/wiki/Japanese_Brazilian it's a reliable post, and also there are many links for references. Now its time to have a look into the Brazilian story in Japan through some data captured from the Japanese census.
Data Source
All the data used in this work was captured in this website: http://www.e-stat.go.jp/SG1/estat/ListE.do?bid=000001034991&cycode=0
This website has all the work made by the census on all sort of counting. And if I have more time in the future, I will certainly explore these data in a wiser way. I can't imagine all the interesting and fascinating facts sitting there just waiting to be dug in. And as we know, Japan is highly developed country from where we can take some useful information. I've chosen the list of foreigners by nationality, age and gender containing in all Japanese prefectures, but unfortunately the newest file they have is from the census for 2010, which makes a huge difference after the 3/9/2011 disasters.
Data Preparation
The data that I acquired was a little bit confusing for me. First I stared at it and decided to open up directly in SAP LUMIRA, which was a good idea for realizing that it should be cleansed up a little bit.
The best way that I found that would work, was making all my attributes label and then put the data under them. The above screen shows how the spreadsheet looked like after some work on it. Around 22420 rows.
SAP LUMIRA
The fun part was in SAP LUMIRA, finding out that all the effort in looking for the data and cleansing it up worth. After the import of all my data, it was time to give some meaning to all that columns and rows.
First step was creating the Geography Hierarchies with Japan city names and unfortunately SAP LUMIRA couldn't recognize the prefecture or the city of many places in Japan. And they are not only small cities, I was surprised to see that major cities, with a huge economic relevance for the country was not recognized. So it may be fixed for the next release or some update somewhere. My maps will be missing information for these cities and prefectures.
My first visualization was made to discover the ranking of nationality by population in Japan. Since the data acquired by the census only reports the ten largest communities this was really easy to understand. It seems that on 2010 South Korean and Chinese were the biggest community in Japan, despite all the historical and current issues between these countries, it seems that a lot of Chinese and South Korean found in Japan a welcoming land. I used a column chart to demonstrate this data:
Before I narrowed my research on Brazilian population, I wanted first to locate the general immigrants population by gender. This one is also a column chart that demonstrates the huge gap between the amount of Chinese, Thai and Philippine ladies comparing with the men from their countries. For the US and UK citizens however, the men population amount is bigger. What would make a good business chance, aiming in the female community of these 2 nationalities?
Now looking in the Brazilian population only, I used a column chart (that seemed to work better for the type of data I was using) in a horizontal mode. Aichi prefecture turned out to be where most of Brazilian that are living in Japan relies on. Aichi prefecture has around 7.4 million inhabitants and its capital city is Nagoya (in this post represented separately), Aichi has 38 cities (including Nagoya) and many big Japanese companies (Like Toyota) are headquartered there. This fact also explains why many Brazilians are living in Aichi. I've never been there, but some friends told me that Aichi has many groceries that offer Brazilian products for the ones that miss the flavor of the country, if you think that they are not enough, Aichi would be a great place to open a business aiming the Brazilian community.
The same insight but now demonstrated with percentage and a donut chart. The data for the cities was ranked with the top 10 cities, using this function inside the population measure. This tool was great, because before, I was using a filter, which also works, but the rank thing gives me what I want to see much faster and with a reliable calculation.
The female population of Aichi is slightly higher than male, but in Shizuoka the game change a bit.
Now, looking into some data about age, I went to compare the kids population from 0 to 14 years old. I'm curious to know if these kids are being prepared to stay in Japan or to come back to Brazil...
For the both graphs above, it comes to light that also there are many young adults in Japan. It would be a good source of research to find out if these young community grew up in Japan, or left Brazil looking for a better paid jobs with or without their parents. I agree that in the 90's and early 2000, Brazil grow were very low and not so many opportunities have arose to the people in college age, but now, even the poorest may have the chance to go to college here, since government is offering a huge amount of scholarship, cutting taxes from private colleges so they can also offer some benefits.
The elderly population seems to be not so big, since only few went back to Japan after the first arrival here a century ago.
This heat map gives us an interesting insight about which age range is strongest represented in Japan separated by gender. Between 25 to 49 years old, both male and female are the biggest community. Again, comparing with Brazil, the young in this range would have more opportunity than the mature.
To me, this tree map view is one of the simplest way to show majority. Easy to understand and easy to build. As stated before, Aichi prefecture has the most of the Brazilian population in Japan, followed by Shizuoka, Gunma and Hamamatsu.
I couldn't use this Geographic Bubble chart as I wished because some cities are missing from SAP LUMIRA side. When it is fixed I want to organize my data with regions that allow me to see the map divided my the 8 informal regions.
In this heat map I wanted to give final insights including all nationalities that were available in the census data. In the picture above we have the population of Chinese and Korean highlighted in the cities of Tokyo and Osaka. Also few (comparing with the last two) Philippines and Brazil, for the prefecture of Aichi.
It seems to be that the women are taking the control, specially in Tokyo. Which would certainly make the things better if followed worldwide.
Twenties and early thirties are the majority in the country, when looked over all nationalities.
And this Geo Chart, that shows the ten nationalities most present in Japan. The label for China disappeared, but the one for South Korea is there. :smile:
And to close the work a cloud of tags. It's a more informal way to show majority.
Conclusion
As I said in the beginning, this work is not intended to be something complex in any way. It is just a modest post to show how I got along with SAP LUMIRA, and show how fantastic and easy can be to handle all the data available. I really had a lot fun working on this post. For the data I have used here, other people or association could use to make people lives better. For Brazil, I've seen many young coming back from Japan without school education, college or even a perspective to get a job and I think there are many ways to prepare these young to suffer less impact when the go back to the homeland.
Data sitting still in a Database, without meaning, is just data after all. SAP LUMIRA transforms data in a show of insightful information. Thanks for reading!
- SAP Managed Tags:
3 Comments
