API Economy - An Introduction

What Is An API?

API stands for Application Program Interface. APIs allows programmers to write a mobile app to look up your monthly gas bill from stored in the cloud database. In other words, the cloud database provides database information through the API.

How Does API Work?

So how does the gas bill  app access the data in the database? Through the database API. The API is a clear set of software functions (provided by the database) to allow another program (the address book app) to retrieve customer information. Simple examples of API functions can be get_bill().

 

Let's assume that PG&E (a utility company) has 4.3M gas customers. The information on the 4.3M customers are stored in a database for 1. Billing (I need to send a monthy bill to customer Nat Gus) 2. Service (customer Nat Gus has 2000 SF house, has a gas water heater, gas stove range).

So far, this example is one where the API user (address book app) and the API provider (the customer database) is in the same company PG&E. But the API (to the customer database) can also be MONOTIZED. That is, you can sell the right to use the API.

Another Example - Making Money from API

At the business level, API allows you to finally make money that was not possible before API. it allows companies with 1. data (customer data, weather data) or 2. service (send us data, we return insight) to SELL. Another way of making money from API is to allows access to your PLATFORM (Amazon, Alibaba, eBay) and you charge a transaction fee. Let’s look at the ways:

Data To Sell Via API And Make Money From Your Data

The WeatherChannel has been collecting weather information around the globe. You are an insurance company that want to calculate th risk of flood in Oslo. The WeatherChannel will sell the insurance company access to the weather data via. Once the payment is made from the insurance company to the WeatherChannel, the WeatherChannel will give. The insurance company an API KEY, and the API documentation. 

Service To Sell Via API And Make Money From Your Software/Know How

You have developed an algorithm to predict who will drive recklessly and cause an accident. You want to “sell” this algorithm. One way to do this is to provide a API to your algorithm. Insurance companies will pay you money, you provide an API KEY and API documentation, they send data (age, race, family incoming, zip code) etc, the you can figure out if they will cause an accident. Service can be more than algorithms - it can be a team of analysts in India doing research.

Grow The Platform API And Make Money From Transactions

You have an e-commerce platform. On this platform, sellers can post goods for sale. Buyers can shop and buy. You want to grow this platform - the more sellers you have, the more buyers will come. You provide a FREE API to verified sellers for them to post their goods for sale. When an item sales, you get money. 

Conclusion

An API enables business to safely share or sell their data on the internet. 

Seeing New Insights From Data - Thanks to GPUs

Geo-Data + Social = New Business Intelligence & Insights

Whereas much of the enterprise business analysis has been focused on RDBMS and Big Data sales transactions to test hypothesis and create reports, the adoption of geo-data on business transactions is an area of huge opportunity. Several companies and industries have already adopted geo-data and are reaping financial benefits. For example, UPS is using geo-data to optimize truck delivery routes, aiming for as many right turns at traffic intersections as possible. This will result in an anticipated $50M saving per year. 




Enterprise Insights Exploration of Geo-Data + Social

If you looked at your favorite social media apps, you will find that they want to track your location. These apps take your location—combined with what you are doing, how you feel, who you are with, and why you are there—provide invaluable and difficult to obtain insights about you. For example, if on January 21, 2017, between 2PM-8PM, you were at location 37.79° N, 122.39° W, and you tweeted that you were feeling happy and civic, you were probably part of the Women’s March in San Francisco. Hence, a certain marketing profile can be built up on you for target marketing.

Enterprise Insights   Exploration Hampered by CPUs

A business analyst, seeing the value of geo-data, wants to perform an ad-hoc query. She has data from Women’s March with an estimated 4 million marchers nationwide. She can query who was at the start location of the Washington D.C. March (38.88° N, 77.01° W), at the starting time (1:15 PM EST), and Tweeted or Liked positively. This is the profile of a enthusiastic, conscientious person. The analyst can also query who was at the end location of the March (38.89° N, -77.03° W), but at the starting time of the March— perhaps a supporter or reporter.  Acting on the speed of thought, the analyst wants access to billions of rows of data, to draw a perimeter of the map to localize around the start of the March, focus on the start time, and filter by contextual data. And after that, try again with another set of criteria so that she can constantly refine her hypothesis to reach a conclusion.  But currently, each click will cause minutes or even hours of calculations before results are seen. This is due to the nature of CPUs – limited number of cores, memory speed, and the types of instructions it excels at.

GPUs to the Rescue of CPUs

Querying a database requires processing cores and fast memory. CPU based servers is limited up to only 22 processing cores and fairly fast memory. CPUs need to be clustered together to be able to serve the queries of billions of rows of data. Another type of processor, called GPU, has thousands of cores and very fast memory. The cores in the GPU process data in parallel and pass data extremely fast to memory. GPUs are so powerful that a single GPU server can sometimes replace multiple clusters of CPU servers. GPU can save money, reduce labor, lower energy consumption, and reduce space over CPU.

G-DB Harnessing the Power of GPU for Map Point Exploration

Whilst GPU is a great match for looking through billions of records in milliseconds, a database optimized for GPU is needed. That’s where G-DB comes in. G-DB offers two synergistic products – G-DB Server and G-DB Visual. G-DB Query is a GPU optimized database. It is an in-memory, columnar data highly optimized to harness the power of thousands of cores in the GPU.  Every SQL query that you submitted is broken down and re-targeted to run in parallel on thousands of GPU cores. That’s how we are able to return queries on billions of rows in milliseconds.  But the magic doesn’t stop there. Synergistically, GPU is also ultrafast at drawing the output of the query results. This is where G-DB Visual comes in. It renders the results of your queries immediately – so that you can use your eyes to help you brains to discover insights immediately.

Conclusion

Transaction, geo-data, and social media combined will enable insights into people not possible before.  Processing billions of rows of this type of data will be slow and/or expensive on a CPU based system, making this valuable data inaccessible. But GPU based systems, like G-DB, can handle this type and size of data with ease. With G-DB, not only can you gain insights at the speed of thought, you have ultrafast high fidelity visuals to match.

A Look At Google Cloud : Enterprises Shifting Traditional RDBMS To Cloud

The latest talk on BigData has made traditional RDBMS relegated to a dark corner. But my experience with database is that RDBMS still plays a big role in enterprise operations : either SMBs that don't need the power of BigData (the barrier to entry can be high), or large enterprises are perfectly happy with traditional Data Warehousing/ETL/BI flow.

There are a large number of cloud RDBMS offers : Amazon AWS, Microsoft Azure, and Google Cloud. Google Cloud SQL offers an easy way to quickly setup a MySQL database in the cloud. Without much instruction reading, I was able to intuitively set on up in 5 steps.

1. Create A Google Cloud  Project

All Google Cloud project starts with a Project name. This is the master that can be used to control all infrastructure instances. In fact, it has its own shell (called gcloud) that can be used to programmatically  spin up, shut down, clone, scale infrastructure. For now, I will focus on just creating a project called "projectsource".

2. Create A Google Cloud SQL Instance

A SQL Instance is an easy way to start a  MySQL server. Nothing is really required of the user. I chose default settings for all settings except password.

3. Create A Google Cloud Bucket, Import SQL Query Into Bucket

My SQL database table schema creation and population Python script was written and debugged on my laptop. This could have have been done easily in Google Cloud, but that's a topic for another day. After creating a Google Cloud bucket, I import my SQL query into the bucket.

4. Import SQL Query From Bucket to Google Cloud SQL Instance

Once the SQL query is in a bucket, it can be easily imported into the Google Cloud SQL instance.

The only modification I had to make to the SQL query is that I must "USE" and database. So in the SQL query file (before it was uploaded into the bucket), I  inserted at the beginning of the SQL file  "CREATE DATABASE dbSource; USE dbSouce;"

5. Connect Google Cloud Cloud Shell To SQL Instance and SQL Query To Your Hearts Desire!

Pressing the "Google Cloud" button at the button will invoke the Google Cloud Shell. This is where you will be prompted for your MySQL password.

Conclusion:

Google Cloud SQL enables a quick setup of a MySQL in the cloud, a easy way to upload a SQL query file, then use the full power of SQL in the cloud.