Second Bangalore Docker meetup @ Red Hat

We had second Bangalore Docker meetup @ Red Hat Office on 22nd Feb’14. ~65 people registered for the event though only 40 – 45 people turned up, which I think is pretty good.

Introduction by attendees

We started the session with a round of introduction from everyone. Not many attendees used Docker before, so I hope the sessions were help for them.

Vijay Bellur started the session by giving an overview on of Red Hat and Docker and how Red Hat is planning to integrate Docker with GlusterFS, OpenShift Origin and RDO (Red Hat OpenStack).

Next session was from Vamsee Kanakala  Introduction to Docker. It was very information and gave very good overview of Docker

Venky Shankar then gave quick and nice Demo. He covered basic Docker commands like how to get the image, start a container, commit it etc. After Venky’s talks Gawtham Sadasivam shared his experience to create a docker file.

Last session was from Devashish about setting up logstash inside docker.

At the end we had some snacks and open discussion. Thanks a lot for Docker team for sending Docker T-shirts for speakers and stickers for everyone.

I hope everyone learnt something new during the meetup. Hope to have next meetup soon.

25. February 2014 by Neependra Khare
Categories: docker, geek, learning, redhat | Tags: , | Leave a comment

Running GlusterFS inside docker container

As a part of GlusterFS 3.5 testing and hackathon, I decided to put GlusterFS inside a docker container.So I installed docker on my Fedora20 desktop

$ yum install docker-io -y
$ systemctl enable docker.service
ln -s '/usr/lib/systemd/system/docker.service' '/etc/systemd/system/multi-user.target.wants/docker.service'
$ systemctl start docker.service
$ docker version
Client version: 0.7.6
..
Server version: 0.7.6

and then started a Fedora container

$ docker run -i -t mattdm/fedora /bin/bash

Once I am inside the container I installed GlusterFS packages

bash-4.2# yum install glusterfs glusterfs-server -y

And then tried to create volume

bash-4.2# /usr/sbin/glusterd
bash-4.2# gluster volume create vol 172.17.0.3:/mnt/brick/ force


but I got following error:-

volume create: vol: failed: Glusterfs is not supported on brick: 172.17.0.3:/mnt/brick.
Setting extended attributes failed, reason: Operation not permitted.

From above error it looked as setting up extended attributes are not supported, which is a basic need to use GlusteFS. So I tried to test them manually. I was able to set extended attributes in user namespace but not in trusted namespace.

bash-4.2# yum install attr -y
bash-4.2# setfattr -n user.foo1 -v "bar" a
bash-4.2# touch a; setfattr -n trusted.foo1 -v "bar" a
setfattr: a: Operation not permitted

With some internet search I figured out that CAP_SYS_ADMIN is needed for setting up extended attributes in trusted namespace and to get that inside docker we need to run an image with –privileged=true option like

$ docker run --privileged=true -i -t mattdm/fedora /bin/bash

With that I was able to create the volume and start it

bash-4.2# gluster volume create vol 172.17.0.3:/mnt/brick/ force
bash-4.2# gluster volume start vol

But when I tried to mount the volume I got following error:-

E [mount.c:267:gf_fuse_mount] 0-glusterfs-fuse: cannot open /dev/fuse (No such file or directory)

this turned out to be image specific problem, which I am using (mattdm/fedora). I had to mknod for /dev/fuse

bash-4.2# mknod /dev/fuse c 10 229

and after that I was able to mount volume.

On Fedroa 20 for docker version 0.7.6 the default storage driver for docker is device-mapper on which extended attributes are supported. AUFS storage driver does not support extended attributes as of now.I have tried with btrfs storage driver with docker 0.8 as well and was able to use GlusterFS. To use btrfs storage driver, we need to start docker daemon with following command :-

$ docker -d -s btrfs

Above will only work if Docker is running on a btrfs partition already prepared by the host system.

16. February 2014 by Neependra Khare
Categories: docker, geek, gluster, learning, redhat, rhs | Tags: , , , , , , , | Leave a comment

Is increased data collection and information sharing improve or damage 21st-century life?

In last week of our course Understanding Media by Understanding Google we have to answer question similar to mentioned above. Here is my take on that:-

We love convenience !! We are always willing to pay something extra for it. The payment we make may, not be the direct one. Lets say you want to have pizza at home on a rainy day and you following options :-

  1. Cook yourself
  2. You like pizza from shop A but it does not do home delivery
  3. Order a pizza from the shop B,  which provides OK kind of pizza but do free home delivery.

For simplicity lets assume cost of each of the above option is same. Given the above choices most of us, including myself going to opt of the 3rd options because :-

  • We’ll get pizza at home without doing addtional work.
  • No need to take a break from on-going activity like watching football/baseball etc to cook or drive to the shop.
  • No need to pay extra money.

By ordering pizza from shop B, we are compromising the on the quality and bit on our taste. This is an indirect price we are paying for the convenience. On the internet lets try to compare this indirect price with :-

  • Increased data collection and sharing by internet giants.
  • decrease in anonymity.
  • loss of personal touch with friends, relatives.

In 21st Century, due to high demanding work, traffic, choices etc; I think it is worth paying some indirect price for the convenience we are getting. We save time, money which are two of the most important things which improve life in 21st Century. As mentioned by Jeff Jarvis in Economist Debate on Social Networking  “We are sharing for good reason—not because we are insane, exhibitionistic, or drunk. We are sharing because, at last, we can, and we find benefit in it.”

Until we are just paying limited indirect price we would be able to make perfect balance in our public and private life. The problem starts when we become greedy and start ordering pizzas from some low standard shops which charges less money and deliver at home. This would eventually kill our taste and we would forget the original joy of having the pizza. Similar thing applies on what we share on the internet. We just  be careful on what we share !!.

25. October 2013 by Neependra Khare
Categories: #googlemedia, coursera, learning | Tags: , , , | Leave a comment

Is our constant urge to use mobile device or do google search good ?

In second last week of our course Understanding Media by Understanding Google we have to answer question similar to mentioned above. Here is my take on that.

Lets say our referred in the question is me. While at work or otherwise when I am engaged with my primary activity I do Google searches and browse the internet to somehow enhance and deepen my understand with it.  Here is my observation :-

  1. About 70% of the time that would help me to finish my primary activity or job quickly like finding howto instructions to setup a development environment etc.
  2. For rest 30% of the it can’t help like stop a crying a baby.

Because of our high success rate we hope to find every answer on the internet even when our primary activity has more to do with personal relations, friends. Other than high success rate we get answers within few minutes, even if we have to skim through multiple search results. This increases our expectation at all fronts of life. And we end up :-

  • doing multitask.
  • having urge to be connected with internet all the time.
  • not paying attention to content of one link as we want to skim through more.
  • responding to non-urgent stuff urgently like FB post, chat pings.
  • not paying enough attention to the person in front of us.

As mentioned in Joe Kraus video  our mind is not designed to do multitask as computers and we are 40% less productive if we do so.  I think this true in most of the cases. Only is very few cases context switching between multiple things/work helps and we have to careful when we do that.  On the contrary all the internet giants are trying to make internet as part of everyday. They are urging you keep connected all the time. Because this means business as mentioned in Nicholas Carr, “Is Google Making Us Stupid? article.

Clearly the choice is ours and we shouldn’t use the technology at the cost of personal relation.

21. October 2013 by Neependra Khare
Categories: #googlemedia, coursera, learning | Tags: , , | Leave a comment

Comparing YouTube’s impact on politics, culture, news in India

One more week is gone and this week as part of assignment for Understanding Media by Understanding Google we need to explicitly rank YouTube’s impact in these three areas – politics, culture, news and then explain our choice for the area we believe to have been most affected. Here is my take on that:-

I live in India and as per my opinion YouTube’s impact us (in India) in following areas in descending order:-

  1. Culture
  2. News and Politics

Culture is a very broad term. It can be considered as group of people or society bound by  :-

- ideas
- customs
- social behavior
- education
- arts

Now if in every second, more than one hour of video is uploaded on YouTube  and with over 6 billion hours of videos watched each month , a niche culture is getting created every part of the World. Again this aligns with Google’s goal to serve the long tail. India is no exception here. If we look at the YouTube channels for India then we’ll see following :-

- Best of YouTube
- Featured
- Music
- Comedy
- Films and Entertainment
- Gaming
- Beauty and Fashion
- From TV
- Sports
- Automobile
- How-to and DIY
- Tech
- Science and Education
- Cooking and Health
- Causes and Non-Profit
- News and Politics
- Lifestyle

Inside each of these categories there are many sub-channels. From the definition of culture we can consider each of these categories as a type of culture. I am sure each one of us would be part of one or more types. I personally visit YouTube for Music, Tech, Science and Education and News often. As far as I know most of friends and family members visit channels from one or more categories.

From the above I can say that YouTube’s imapct on culture in the most.

Also from the above list  News and Politics is a sub-category and can keep them in same bucket. If we look at the trend dashboard for India :-
http://www.youtube.com/trendsdashboard

we do not see many people are interested in following News and Politics on YouTube.

13. October 2013 by Neependra Khare
Categories: #googlemedia, coursera, learning | Tags: , , , | Leave a comment

Cruise to the Coast, 5-6 Oct’13

Last weekend I did Cruise to the Coast ride with CAM.  We cycled through Bhagamandala – Bekalfort – Kasargod – Jalsoor  and covered ~130 kms. We started from Bangalore on Friday (4th Oct) night and reached Bhagamandala at 5 AM on Saturday. Then after couple of hours of rest we started cycling to Bekal Fort. We followed http://goo.gl/maps/dDFtN route.

100_1756

On the way we saw many waterfalls. Both sides of the roads there were just trees. I maintained distance from other cyclist most of the time. It was like I was alone on the road. All the nature beauty was just for me. While cycling I can see yellow butterflies on the roads, like they were welcoming us. I used to see and play with them in my childhood. I almost forgot that there are such butterflies.

Most of cycling we did in Kerla state and I must mention that the people/govt. has tried to keep it green and clean. Some houses on the way were very beautiful. They had very good wood work. Some houses even had their own water well. I know this sound stupid to mention but I have seen them since ages.

On they way be Bekal Fort it started raining. I cycled around ~20 kms during that. It was such a great feeling. I even tasted the rain water which got mixed with my sweat. It was sweet ..:).

IMG_20131006_110539

I reached at Bekal Fort ~3 PM. After spending some time there we headed to our hotels. Second day we started at 7 AM cycled from Kasargot to Jalsoor, which was around 50 kms. We followed http://gooaps/oOx8c route.

We reached to Jalsoor around 11:30 AM and started our journey back at ~12:30 PM. We reached Bangalore ~10:30 PM.

During the ride people greeted us, kids have us high five. Kids kept asking “Whats your name” and “Where are you going”.

Overall it was fun ride.  I made some new friends and hope to keep in touch with them.

10. October 2013 by Neependra Khare
Categories: Uncategorized | Tags: , , | Leave a comment

How relevant are the ads, we see ?

As mentioned in the previous blog post I have been doing Understanding Media by Understanding Google course on Coursera and as part of second peer graded assignment I have to log the next 20 ads I encounter from Google, whether as the result of a search on google.com or through one of its partner sites and summarize what those ads get right about me and my interests, including at least one example, and what you believe they get wrong.

So I logged them and here they are:-

https://docs.google.com/spreadsheet/ccc?key=0AlVUJuil1-urdFVPNGNWYUo4cHVfM1RxblByenRYa0E&usp=sharing

and to summarize.

I  :-

- am male
- am married
- reside in Bangalore, India
- work for a Linux based company
- Interested in cloud technology
- have been searching for training in hadoop and a cars using Google
- have been searching for books in flipkart
- fly often
- do banking
- buy consumer goods

From the above list following advertisements are right about me and my interests :-

- Consumer Electronics
- Training
- Air Travel Agent
- Online retailer
- Banking
- Cloud Product
- Online yellow pages
- Car dealer

Following advertisements are somehow interest me :-

- Hardware Vendor (Dell)
- Beverages
- Tyres
- Real State
- Greetings, Gifts

Following advertisements do not interest me and are wrong :-

- Matrimony (as I am already married)
- Forex Trading

If I look at the advertisement more closely then just on one page I am seeing ads about real estate, travel, retailer, matrimony, gifts.  Just look how commercial a web page has become. And on top that we now see ads based on personal needs, preferences. This is what can be called as serving the long tail.  Web companies are collecting more and more data about us and we giving them for free. But the web companies are charging their advertiser aka our seller.

07. October 2013 by Neependra Khare
Categories: #googlemedia, coursera, learning | Tags: , , , | Leave a comment

Is Google’s intention to digitize all the world’s books a good thing or a bad thing, and for whom?

I have been doing Understanding Media by Understanding Google  course on Coursera and as part of first peer graded assignment we have to write about  ”Is Google’s intention to digitize all the world’s books a good thing or a bad thing, and for whom? “.  Here is my take on it :-

Imagine if every book in the world is digitized then we can:-

  • search books as we search publicly accessible web pages today without paying any fee. This will enable us to access books from anywhere with different devices, choose best possible book etc.
  • give access of same knowledge base to everyone irrespective to country, income level.
  • increase the visibility and life span of books by linking/tagging them and creating conversation around them.
  • collaborate more easily, which can put seed for new ideas.
  • get access to out of print books.

As mentioned by Jeff Jarvis in ”The Google Times”,  that we need to kill books to save them and above points support it. Other than the authors of out of print books who want their work to reach more people, a general user like me/us would get benefited by digitization of books from Google.

But wait !!! everything comes at a cost. Google is a publicly traded company and it has to make profit to keep its investors happy. As per Siva Vaidhyanathan lecture, “Google, Digitization, and the Future of Books” and from some his other work :-

  • neither Google nor the publishers guarantee to protect user privacy when using the Google Books site.
  • we would loose the personal touch which we get librarians.
  • digitization of books can lead to privatization of libraries.
  • the quality of the scanning is not good enough.
  • with digitization of books Google is trying to make its search better.

By making its search better Google is effectively increasing revenue. Also there is no guarantee that based on user private data, Google would not show sub-optimal results with search; to benefit its Ad sponsors. This is bad for the user. After digitization of a book Google can sell it online to make money from someone else’s work, which is bad for the original author.

02. October 2013 by Neependra Khare
Categories: #googlemedia, coursera, learning | Tags: , , , | 1 comment

Bangalore::Hack 2013 by Sequoia Capital

I participated in Bangalore::Hack 2013 last week and was planning to do a blog post about my participation. So I started to collect the relevant link to put on the blog post and came across this :-

http://yourstory.in/2013/09/sequoia-capital-bangalore-hack/

and saw the mention about my work.

4) Government information aggregator – This one is close to my heart; a tool that aggregated government information to empower journalists with data.

I am very happy to get comment like this from Yourstory. Thanks to Yourstory for covering it.

Though I had not given any name for my project but lets call it as Government information aggregator  for now. So here is the pitch about my idea:-

With this tool I would like to help journalists and socially active people who are trying to make difference at ground level. Being in metros we have access good new papers like The Hindu, good internet access to visit different news sites. But if you go any tier II or III cities and read the local news paper there then we do not see similar quality content. This is because journalists or people do not have access to technology by which they can backup their stories with relevant data.

So the idea here is to get the data from government website like data.gov.in and list of policies government has implemented over time. And then try link them together to see what difference those policies have made.

Lets take an example to literacy rates of different states in India over different census. It has varied state by state. This can be visualized using the app the I wrote and made it online during hackathon:

http://spark.rstudio.com/nkhare/banghack/

If we can relate the percentage changes in given state with the policies it has implemented over the years then we can get a sense about what caused that percentage change. I did not figured out exact way to relate the change with policies but I did google query for policies from sites sites like gov.in, nic.in and list down the result as shown in following screenshot:-

MP_Education

Education is just an example. We can get the similar data from sites like data.gov.in for health, food producation, manufacturing etc. For now the user have to look at the listed policies manually but this narrows down search.

We talked about state wise here but this can be extended to district level. If we can visualize percentage changes in different sectors for a given district over the years then journalist or others can pick up that data and ask questions to local governing bodies. The data-set would be picked from the government websites, so there is no question about authencity. We just need to visualize them and share.

I have written the app using R and Shiny for which I recently reviewed a book. The book  should get publish in a next month.

I never heard about Sequioa Captial, till I came to know about Bangalore Hackday sometime last month. I came to know more about it during the Hackday. And since last week I am reading more about it through the books I am reading about Google for my coursera course Understanding Media by Understanding Google .

I participate alone using team name “Shadowfax”. It was great event. I learnt many things, met interesting people and saw wonderful projects. I hope to participate in events likes this in future.

22. September 2013 by Neependra Khare
Categories: Data Analysis, geek | Tags: , , , , , , | Leave a comment

GlusterFS presentation @LSPE-IN, Yahoo!

Last  Saturday on 14th September’13 I gave on GlusterFS presentation at LSPE-IN. The title for the presentation was Performance Characterization in Large distributed file system with GlusterFS . Few days before the talk I looked at the attendee list to get  the feel of the audience. I felt  that not many of them have used GlusterFS by now. So I decided rather than covering lots of performance part I should cover the concepts of GlusterFS and talk about the performance challenges which can be there by design like file-system in user-space etc.

I am glad I did that as there were only 4 folks from ~70 attendees who have used GlusterFS before.

By going through the concepts the audience were able to compare GlusterFS with other solutions in the market. They asked interesting question about hashing algorithm, replication etc.

IMG_0649_v1

I attended the meetup till lunch. Other than giving the presentation I attended a keynote on Challenges in scaling cloud storage  by Srivibhavan (Vibhav) Balaram in which he talked about the challenges in cloud storage and  QoS in cloud environment. I also attended “Scaling using event driven programming with Perl, A tutorial”  Aveek Mishra.

This was again a very informative and well managed event. I’ll look forward to attend and present in future meet-ups.

16. September 2013 by Neependra Khare
Categories: gluster, performance | Tags: , , , , , | Leave a comment

← Older posts