Kids and STEM

BBC Micro running The Crystal Connection from Flickr by trevkg under CCWhen I was 12 years old, I was introduced to the BBC Micro and I used to program in BASIC (Beginner’s All-purpose Symbolic Instruction Code) with the help of the BASIC language manual. That’s what inspired me to get into programming and computers. I used to write/play games, utilities etc. On the right is what a BBC Micro looks like.

Forty years of removable storage from Flickr by avaragado under CCAll the data (games, programs etc) used to go in a 5.25 inches floppy disk (the biggest media on the left) which holds around 2 MB of data. Now a days a single picture taken from an ordinary digital camera is much more than 2 MB. It’s so much fun remembering the old days.

I am trying to do the same thing with my kid (7 years) also. Get him interested into Science. When ever I complete one of the basic circuits, I show it to him and explain it using the concepts which he already knows. He might be not be able to build them today, but helps him build curiosity on how different things. Here he is holding the Arduino Uno and the Induino. Whenever I get a new piece of hardware, he is more excited than me to know what it is and what it does. I have to scream at him not to touch them with wet hands, so as to not short the component :)

Prajval with the Arduino and Induino KItsToday I did build an Infrared-Receiver Circuit with the Arduino Uno R3 and the Infrared-Receiver Module which came with the Sunfounder Arduino Sensor Kit. The IR sensor will receive IR light from a remote control and then turn the LED connected to the Digital Pin 13 of the Arduino board on and off. It’s as simple as that.

The way I decided to explain him was taking the analogy of a Television remote control which emits the IR light, then the receiver in the Television gets it and takes the appropriate action. He was very much excited to know how a TV remote works and also the circuit in action.

Till now, I had been mainly into programming which is nearly invisible and it’s difficult to get kids interested into software. And also software is becoming more and more complex to write. The BASIC language which I mentioned earlier, had a very simple syntax and fun to program with. With the recent IOT I had been doing, it had been easy to show him the different circuits physically and map them to what he already knows like a Television remote.

With constant reminders from various directions that today’s kids are getting behind in STEM (Science – Technology – Engineering – Maths), it’s equally important to gets the kids excited about them. As I mentioned above, my kid might not be able to build a circuit today, but he now knows how a Television remote works. I am very happy about it.

I did order a CanaKit 4 Wheel Drive (4WD) Robot Platform for Arduino and anxiously waiting for it to arrive to build a robot with it. Planning to build a robot with a couple of sensors like obstacle detector, line follower etc. My son and myself are really excited about it.

If your kids are a bit old, I would very much recommend to buy something like a Raspberry Pi 2 for them or else buy one for yourself and explain it to him how it works. The Raspberry Pi 2 costs 35$ plus the cost for accessories like a power adapter, USB cable, memory card etc. It’s worth the price.

Induino, the Indian version of Arduino

Before looking at Induino, lets take a look how the open source works. In the case of Linux there is GNU Linux which is Free and Open Source Software (FOSS). Anyone can take the base code and make improvements to the base GNU Linux and release as a derivative. There are more than 100 different flavors of the Linux. Fedora, CentOS, RHEL, Ubuntu are a few of them and there are companies like Red Hat, Canonical which support them. They make money by providing commercial support, training, consulting etc.linux-distributionsThe same can be applied for the Big Data softwares also as shown below. There are multiple projects under Apache which are FOSS. Companies like Cloudera, Hortonworks, MapR, Microsoft take the base code from Apache and create their own distribution. All of them try to improve the base Apache by providing better documentation, performance improvements, better usability, bug fixes etc.big-data-distributionsBecause of the open source nature, there are lot of companies around Linux and Big Data. They make revenues through multiple channels, but not many have a positive cash flow.

The same model can be applied to hardware also. The schematics and the reference designs are available for the Arduino Uno here. So, it’s possible to create to create  a clone exactly like the Arduino Uno or add some additional features to the base and create a derivateHere is a nice article from Arduino on the same.

Induino R3 is one such derivative of the Arduino Uno from Simple labs. Here is an overview of the Induino R3 and here is the user guide for the same. The Induino R3 costs 1,260 INR and Arduino Uno R3 costs 1,575 INR. There is not much of a difference from a cost perspective, but Induino R3 has some additional features. More about the difference here. Not sure how the other board are, but the Induino R3 seems to be water proof and very rigid. More videos on stress testing the Induino R3 here.

Here is a video from the founder of Simple labs about the Induino R3. He seems to be very passionate about what he has been doing.

I am not sure if there are any other similar hardware made in India, but looking at this made me proud and happy and so the post on the same. Here is the Arduino Uno R3 (left) and the Induino R3 (right). The Induino is a bit bigger than the Arduino because it has some additional features.Arduino Uno and Induino I will write a detailed review of the Induino R3, once I get a chance to play with it.

Integrating Apache HBase (Big Data) with Arduino Uno (IOT)

For some time I had been blogging about the intersection of Big Data and IOT. The combination of Cloud, Big Data and IOT provide a lot of opportunities. In this blog we will look on how to integrate Apache HBase with Arduino Uno. HBase is a NoSQL database of columnar type. HBase has got some nice documentation here.

Adding Ethernet Shield to Arduino Uno

arduino-uno-ethernet-shieldArduino Uno by itself doesn’t come with a facility to connect to the internet. So, an Ethernet Shield has to be added on top of the Arduino Uno board. Here we see the Arduino Uno connected to the USB cable on the right and the Ethernet Shield connected to RJ45 network jack on the left. Now the Ethernet Shield has to be mounted on the top of the Arduino Uno for gaining the network capabilities.arduino-uno-ethernet-shield

Shields are expansion boards to provide additional functionality to the Arduino Board. More than one shield can be stacked on each other. Many shield are available in the market. A custom shield can also be built using the Arduino Protoshield.

In the coming blogs, we will look at the Arduino Motor Shield and build some cool things using it.

Starting the HBase and the REST service

HBase provides different interfaces (Java – REST – Phoenix – Protobuf) for the DDL/DML operations. In this tutorial the REST interface will be used. After starting the HBase, the REST server which is part of the HBase installation has to be started. More about the HBase REST interface here and 1, 2, 3.

The REST interface runs on port 8080 and uses the HTTP verbs like POST, PUT, GET and can be  invoked from the browser as shown below. The schema for the table mytable is displayed in the browser here. Because of the language agnostic behavior of REST, applications can be built using any language on top of HBase.

browser-rest-call
Creating a table in HBase and querying it in Arduino Uno

Now a HBase table has to be created using the command create ‘mytable’, ‘metrics’ from the HBase shell. Once the table has been created, the same can be accessed from Arduino Uno. It should be made sure that the Arduino Uno and the HBase are within the same LAN.

hbase-shell

Now that the table has been created,  the Arduino sketch has to be executed to make a REST call. Here (GET, POST) are the complete Arduino Sketch. The HBase-Arduino-GET.ino Sketch is getting the schema for a table in HBase and the HBase-Arduino-POST.ino Sketch is for inserting a row in HBase. Note that the code is not optimized, but is good enough to know the basic concepts and also to get started.

The bottom screen is the Arduino IDE for developing/compiling/deploying the  Arduino Sketches. Bottom right in the same screen is the Arduino Serial Monitor which shows the output of the HBase-Arduino-GET.ino Sketch. Note that the Arduino Sketch does a simple HTTP GET operation to get the schema for a particular table.

arduino-hbase-output

Similarly, the HTTP POST/PUT can be used to put some data in HBase. HBase stores the data as Base64, so the input data has to be converted into Base64 before insert it. The arduindo-base64 library can be used to encode/decode to Base64.

Conclusion

In this blog we have seen how to integrate the Arduino Uno directly with the HBase REST Server, which is a bit restrictive approach because Arduino Uno can’t process much and also there aren’t too many libraries for the Arduino Sketch when compared to languages like Python. Arduino Uno can also be interfaced with an intermediate (through pySerial or Processing) and the intermediate communicates with the HBase REST Server. We will look into the pySerial and the Processing approach in a future blog.arduino-hbase-interfaceIntegrating Big Data with IOT directly  was fun, but had not been straight forward. Big Data and the IOT exist in their own space, may be down the line the different m2m platform vendors will provide some sort of connectors to the Big Data space to make the connection between the Big Data and the IOT worlds a bit more smoother.

In the coming blogs we will look at how to get the ambient temperature from a temperature sensor and put it into HBase at regular intervals.

Prerequisites to get started with Big Data

Very often I do get the query ‘I am familiar with X, Y and Z, is it good enough to get started with Big Data ?’. Old and new construnuctions under Central Station Amsterdam from Flickr by urban portrait in photo pictures under CCThis post is to address the same. Here we will look at what is required to get started with Big Data and the rational behind it.

There is a lot of information on the internet to get started with each one of them and it is easy to get lost. So, the references to get started with those technologies are also included.

Linux : Big Data (the Hadoop revolution) started on the Linux OS. Even now most of the Big Data softwares are initially developed on Linux and porting to Windows has been an after UBUNTU! Tux floats to work! from Flick by danoz2k9 under CCthought. Microsoft partnered with Hortonworks to speed up the porting of the Big Data softwares to Windows.

To get started with the latest softwares around Big Data, knowledge of Linux is a must. The good thing about Linux is that it is free and it opens up a lot of opportunities. Would recommend to go through all the tutorials here, except the seventh one.

There are more than 100 different flavors of Linux and Ubuntu is one of the popular distribution to get started for those who are new to Linux.

Java : Most of the Big Data softwares are developed in Java. I say most, exceptions are Spark has been developed in Scala, Impala has been developed in C/C++ and so on.

To extend the Big Data softwares knowledge of Java is a must. Also, sometimes the documentation might not be up to mark and so it might be required to go through the The Evolution of Computer Programming Languages #C #Fortran #Java #Ruby from Flick by dullhunk under CCunderlying code for the Big Data software to see how some thing works or is not working as the way it is expected to.

For the above mentioned reasons knowledge of Java is must. Basics of core Java is enough, knowledge of enterprise Java is not required. Go through the Java Basics section and the Java Object Oriented section here.

Java programs can be developed with as simple as notepad. But, developing in an IDE like Eclipse makes it a piece of cake. Here is a nice tutorial on Eclipse.

Databases 2 by Tim Morgan from Flickr under CCSQL : Not everyone is comfortable with programming in Java and other languages. That’s the reason why SQL abstractions have been introduced on top of the different Big Data frameworks. Those who are from a database background can get very easily started with Big Data because of the SQL abstraction. Hive, Impala, Phoenix are few of such softwares.

Expertise in SQL is not required. The basics of the DDL and the DML operations is more than enough. Here are some nice tutorials to get started with SQL.

Others : The above mentioned skills are good enough to get started with Big Data. As one gets into more and more at Big Data, would also recommend to look at R, Python and Scala. Each of these languages have got their strength and weakness and depending upon the requirement the appropriate option can be picked to write Big Data programs.

To become good at Big Data it’s required for an aspirant to have a good overview of the different technologies and the above guide mentions what is required and where to start reading about them.

Best of luck !!!

Intersection of Big Data and IOT

Lately I had been blogging about the intersection of IOT and Big Data. They go hand-in-hand. With the IOT devices feeding the data to the Big Data platforms on the Cloud to store/analyze the data and feed it back to the IOT Promise by  mariadelajuana from Flickr under CC devices. One such example is the Nest device bought by Google. Nest thermostat gathers the room temperature, sends it over to the Cloud over Wi-Fi, analytics done in the Cloud and fed back to the Nest again.

This is not something new, but had been there for quite some time. But, the conditions are moving towards more adoption of Big Data and IOT like cheap sensors, Cloud, cheaper and faster internet connections. Here is an article  from ZDNet on ten practical examples on the intersection of Big Data and IOT. For those who are interested in learning more, here (1, 2) are few more references from ZDNet.

I am yet to try it out, but here is a detailed article from IBM Developer Works on building a temperature sensor using Arduino Uno, putting the data in the Cloud and finally visualizing the data in a real time. Here is another article on how to integrate IOT with Azure.