Switching to Bluetooth and new CNB touch sensors

The previous version of the glove used WiFi for communication. However it would be easier for the user to use Bluetooth. For that we bought an Arduino Bluetooth transceiver, which should arrive in a couple of weeks. By using that we can get rid of the big and clunky Raspberry Pi and make the wrist component much smaller.

Using Arduino instead of Raspberry Pi (RPi) creates new problems. On RPi it was easy to run two scripts at the same time, but Arduino doesn’t have an operating system to control multiple programs, instead a single program is executed. This makes it more difficult to simultaneously check if the user has pressed the touch surface and send the gyroscope data.

With RPi we achieved data rate of 100 packets per second. Meaning that the gyro data is sent very frequently. With just a single Arduino Nano the rate dropped to 60–70 packets per second, which is still good for a mouse.

Diagram

Components of the mouse

Above is a diagram showing how the mouse will work. The touch surfaces and gyroscope are connected to an Arduino. In addition, a Bluetooth transmitter is added to allow sending the data to the computer. On the right side there is the computer that receives the packets and processes them to move the cursor. Custom software is required for both the Arduino and the computer. Later on, we can look into implementing HID technology to remove the need to install programs for the computer.

What’s next?

sensors

New sensors from Canatu, designed specifically for us.

We received the CNB touch sensors from Canatu. Next, we are going to add them to the glove and test how we can use multiple buttons instead of just one. We are also going to do some research on Canatu’s competitors in order to compare carbon nanobuds with other flexible touch surface options.

A look at the business and results from interviews

Business

Canatu Ansaintamalli

Canatu’s revenue model

The revenue model points out the flow of inputs and outputs for each player from suppliers all the way to end user. We focus our discussion on the relationship between our project group and Canatu. There are also other interesting parts in the revenue model, such as the type of revenue between manufacturer and end user (which can be for example subscription model or production model), but those are outside of our project’s focus.

The model shows that we are building a demo for Canatu. Canatu is providing feedback to us during the course to guide us into the most feasible direction. Our point of contact with Canatu is their Chief Chemist. The demo would be especially relevant for sales&marketing department, but interesting for the whole company.

The demo is created mainly for Canatu’s direct customers, but also for the end customer. The main purpose of our demo is to showcase the distinct features of Canatu’s CNB film compared to other touch surfaces.

The demo should be funny, interesting and simple enough, so that you can test it with ease, enjoy the experience and understand Canatu’s value proposition.

The demo application doesn’t necessarily need to have extensive customer demand, since it’s promoting the touch surface, not the actual product. But of course the more relevant the demo seems the more attention it will also get from not only Canatu’s direct customers but also from end customers, who can then demand the product that needs Canatu’s CNB film.

 

Canatu_Business_Model_Canvas

Business model canvas for Canatu (Click for a larger picture)

Business model canvas (BMC) lays out the business case behind Canatu’s product, CNB film. Even though in this project course we’re focusing on building a demo for Canatu, it is essential to understand the business that Canatu is building, so that we can assure the demo and its features are serving the right purpose.

In the heart of BMC are Key Partners. Even though Canatu is creating something consumers will use, they are a B2B business. Other businesses, direct customers, are Canatu’s Key Partners and the Customer Relationship with them is both close and personal. A large number of consumers can be reached with only a few prominent Key Partners. Imagine if Samsung would start using Canatu’s CNB film for their curved screens – that should give you an idea how important, and powerful, Canatu’s direct customers can be. Each of these customers can create massive revenue for Canatu. If the relationship is really fruitful, it can create revenue not only for years but potentially for decades.

Canatu’s Key Partners not only bring significant revenue over multiple years, but they can also affect Canatu’s research and development process. Canatu can focus their R&D efforts to serve the special needs of a customer, which supports the personal Customer Relationship. The R&D efforts needed will impact the price for the customer, but in the end Canatu’s price is mostly dependant on the value it can create for the customer. If the product is no better than competitors, there’s no argument for higher price. On the other hand, if Canatu’s CNB film is superior on every aspect after specific R&D efforts made for the customer, then higher price is more than justified.

R&D is naturally the most cost heavy part of the business, but sales and marketing is also notable. Sales and marketing efforts for landing a client, and maintaining the close Customer Relation can be extensive – but they are also productive, since the customer is regarded as a Key Partner that can potentially bring a large share of revenue over a long period of time. In short the cost of acquisition can be high, but also the customer lifetime value can be very high.

Canatu’s customers can be from different industries, but they are served through same means. The actual CNB film is transported via normal logistics (train, truck and ship), and the personal Customer Relationship is maintained through cooperative R&D efforts.

The Value Propositions in BMC are value propositions for end customers, since Canatu’s direct customers make their decisions based on the needs of end customers. So in order to be valuable for Key Partners, Canatu needs to help them serve end customers better.

 

Interviews

After the last blog we have been busy conducting interviews and gathering valuable feedback from outsiders. Like discussed in the business model, our prototype is not necessarily designed with such end-customers as a priority. However, we did get valuable perspectives outside our own bubble of thinking. Below you can find some highlights.

People were interested in using this kind of smart glove to control home appliances / use in conjunction with smart home applications

This is something that we had discussed earlier but we didn’t expect our interviewees to spontaneously bring this up. We suspect that our smart glove is easily associated with emerging and/or future technology and thus the value is seen there. Even though this feedback was quite common, we are still moving forward with the idea of using our glove to control a computer interface. This is due to practicalities as it’s easier and more feasible to demonstrate the unique touch features of our prototype.

The smart glove could be very appealing in professional use because of its capabilities to work in three dimensions.

3D modelling and graphical appliances are not something that we had thought of before. If the tracking is very precise, the use case in such scenarios could be very interesting

There are a lot of practical issues in the current prototype that decrease its value in the eyes of “customers”

As we suspected, it was often noted that it’s not effortless to put on a glove to use something. The vision of being able to interact with appliances effortlessly is not entirely true as long as you have to go pick up the glove, put it on, and start interacting. It’s not convenient to put on a glove to switch channels on television, for example. A similar common feedback was that it’s difficult to see how the prototype brings true value to the user. Many of its applications (a computer mouse, for example) can be replaced with current technologies and products that work just as well, if not better.

WOW

Regardless of its limited practical use, our prototype did attract a lot of positive reactions and attention from our interviewees. In general, people perceived our smart glove as cool and futuristic which is great feedback for us in itself.

Price point varied from as little as 40€ to hundreds and thousands of euros

In general, our interviewees said that “if this does a lot of things and does them well, it’s worth a lot”. The estimates were mostly in the 50€-100€ range for private use, but when the interviewee looked at it from a professional / corporate perspective, the estimates rose to the hundreds of euros. And in one scenario, if a similar product could be used precisely in 3D modelling, its price could be realistically in the thousands.

Key takeaway from the interviews was that we’re creating something that’s is interesting for potential customers. Since the demo can also showcase the distinct features of Canatu’s CNB film we are on the right track for creating both a successful and meaningful demo!

Using the glove as a mouse

We succeeded at connecting the gyro sensor to an Arduino and getting some data out of it. However, the raw data was difficult to understand, but finally a site was found telling how the data is turned into sensible values. The gyroscope data was converted to degrees per second and accelerometer data was converted to meters per squared seconds.

This is where the old demo of a smartphone controlling the computer mouse became useful. By connecting the Arduino to a Raspberry Pi, it was possible to write code that sent similar data to the server program on a computer to tell it to move the mouse based the angular speed data from the gyroscope. We chose to use the Z axis for moving the cursor left and right and X axis for vertical movement. This felt much more natural than we initially thought. We planned to use the accelerometer to detect actual distance the glove has been moved and translate that to cursor movement, but keeping the hand still and tilting it feels easier.

Gyroscope axes. Image source: https://cdn.sparkfun.com/assets/9/9/3/f/b/5112d375ce395ff927000002.jpg

As the gyroscope was working, it was time to try to build a simple prototype that can be put on the users hand. We found a piece of cloth to hold the Raspberry pi around the user’s wrist and an old glove. We attached the gyroscope to the glove and added batteries to make the device wireless. Also, to test how clicking would work with the glove the old touch surface that was used in the very first Arduino demo was added. Again, wiring the touch surface turned out to be very difficult because of the small pins, but luckily Janne came up with the idea that we could use hot glue. The glued wires seem to hold well.

It looks like the main difficulties are related to touch sensitivity. Sometimes, with the current settings, there are miss clicks and also times when pressing the touch surface seems to do nothing. These will be likely fixed by a new touch surface, which we will get from Canatu later on.

Our smart glove even has two USB ports and HDMI. But wait there’s more! Call now and get two additional usb ports and a 3.5mm audio mini jack port!

Making progress

Foreword

Our last week’s presentation went really well and it seems that our progress seems to satisfy both the course staff as well as ourselves. We are making progress in visualizing our business model and we have taken concrete steps to improve our prototype.

Current state of prototyping

Like mentioned last week, we have been looking into a gyroscope and accelerometer and see if we can incorporate them into our working prototype. However, it does take time for the sensor (gyroscope, accelerometer, etc) to arrive from China. Therefore, a smartphone was used to test out how the sensors’ data could be interpreted. The data gathering can bee seen on the video below, but the analysis is still a work-in-progress, as it is not as simple as it sounds to translate raw data into sensible inputs.

Real time sensor analysis

data1

data2

Screenshots of the graphs drawn with Matlab from the data (another take). Red: X-axis, Green: Y-axis , Blue: Z-axis

canatu_plan

A mockup of what we’re going to order from Canatu to test how the buttons would work in hand.

After we have received the custom-made film, Juha will come up with something clever to wire it and make it breathe!

What will be done next?

Last week, we were advised to gather 10-15 interviews from potential customers to support our vision. It is important to gather feedback from outside to see if there is true potential where we think there is. Such feedback will be used to refine the scope and the plan for the final prototype. We met and brainstormed some questions that we deemed fit for purpose and currently we are conducting the mentioned interviews.

Next week, we will have a presentation regarding our current progress and future plans. Results of our interviews are hopefully interesting and we will make sure to publish the highlights on this blog as well!

Bonus

gyroskooppi

We received our sensor from China! Let’s see what we can use it for..

Testing controlling methods

We had a meeting on Tuesday where we prepared the content and slides for the Friday’s presentation. In addition, we discussed how we could improve our control glove. One of the current difficulties is how we could use sensors such as gyroscopes and accelerometers to detect the movement of the glove.

On technical side:

We experimented with gyroscope and accelerometer for controlling computer cursor by creating a mobile app and a server for the computer being controlled. The mobile app sends sensor data to the computer and the server program parses the data and moves the mouse accordingly. We would like to use the accelerometers to deduce hand position for controlling the cursor, but that kind of data is prone to errors due to double integration. Some ideas to reduce error in accelerometer measurements:

  • Multiple sensors
  • Resetting data integration when the movement stops
  • Filtering data for example with Savitzky-Golay filter
  • Using machine learning methods, such as neural networks

Another option is to use the gyroscopes and control the cursor by tilting the hand. This is demonstrated with a mobile phone in the video below.

The controls here are a bit clumsy, but with a wearable controller it would be better.

Next in our plans is to perform some interviews with potential customers. We will ask them, for example, about features they would like to see in such a product and how much they could see themselves paying for a working device.

Week 3 – Project plan, more ideas

This week has been an exam week and there were no lectures or other guided sessions. However, the work on the project never stops!

Remember the glorious sketch of our prototype a few weeks back? Our tech wizard couldn’t contain himself and came up with a great render visualizing the idea even better.

SmartGlove

With this picture, you can get a rough idea of what the prototype could look like in theory. The strap on the wrist includes a bluetooth transmitter that relays the inputs to the device used (computer, smartphone).

A whole new perspective?

Last Friday, we toyed with the idea of utilizing your hand as an input method. This idea progressed even further with adding a gyroscope for full control! We are still not sure if this is feasible in our prototyping but below you can find a YouTube video of such solution.

So the general idea is to use the CNB film on your fingers registering inputs such as clicks, scrolling etc. while the movement of the hand would move the cursor on the screen. Cool? Yes. Feasible? Possibly!

Project plan

One concrete result of this week’s work is the project plan (link) for the spring. After a well-deserved weekend we will begin preparing for our presentation next week!

Meanwhile, these prototypes and ideas will keep growing..

 

 

Second week, making concrete progress!

Visit to the office

After a productive kick-off of the project last week, we were able to keep up the good pace by visiting the Canatu offices and facilities on Wednesday, the 8th of February. Our contact gave us a good overview of Canatu and its work and we got to (literally) get our hands on their CNB film as well. They had a variety of concepts and prototypes to show and it helped us to understand what we’re working with.

We got a very detailed explanation of how the CNB film actually works and it is quite fascinating. Therefore, the team is even more excited to work on this project as we began to understand the whole potential of the technology.

IMG_2676A (mostly) happy team visiting the office and facilities

IMG_2677All machinery is custom to Canatu’s preferences

IMG_2678And this is where the magic happens, the nanobuds themselves are printed on film

At the end of the tour, and after a million questions asked and answered, we were provided with samples of the film as well as a simple touch sensitive button, that could be used to test how the material works in practise.

The first crude demonstration!

Once given the sample button, our tech wizard Juha was so enthusiastic that he immediately came up with a simple demo setup, using some wiring, Arduino Nano, and a couple of LEDs. You can see the result below!

As you can see, we are able to process the input of the CNB film and turn it into something useful. The film not only recognizes touch, it can detect pressure and how far away the finger is from the surface. This working demo has given us a boost in motivation and we are now confident, that we can set the scope of this project to include a functional prototype in addition to refined concepts.

What happened on Friday?

In this week’s session we were encouraged to do some brainstorming to make new ideas for our project. We focused on building on last week’s plans, mainly the control glove and the spherical controller. One idea was the possibility to combine these prototypes. If different nanobud touch surfaces can recognise each other while they are in contact, there can be for example interaction between the glove and the sphere. This is related to the idea of using the touch surfaces with different materials (not just bare fingers).

The little demonstration above showed us that the strength of touch can be measured. This could possibly be used for slider solutions. Also we thought about user friendliness in the programming interface. Possibly our most rousing idea was to combine the glove with a motion sensor, possibly a gyroscope, and use the hand movement to move a computer cursor.

 

Next week we will continue the blog with a project plan. Thanks for reading.

Getting started

Welcome to the Carbon Nanobud blog! Our multidisciplinary team consists of five awesome members:

Juha Eskonen (Computer Science)
Technical wizard that can code your backside to Mars and back, also knows how to actually build electrical stuff

Henri Huttunen (Information Networks)
Jack of all trades, keeping the mess together

Janne Korpela (Industrial Engineering and Management)
If you’re out of ideas, contact this guy

Kaarle Leinonen (Industrial Engineering and Management)
The grandfatherly voice of reason

Mikko Luukinen (Engineering Physics)
Mister Cyborg ready to revolutionize the world with nanotechnology

We started the project with a creative brainstorming session. Our first ideas were

  • Sphere-shaped multipurpose remote controller for home appliances such as tv, lights, curtains and temperature control
  • Camera that has a touch surface also in other parts than just the screen
  • Computer mouse / alternative input appliance
  • Wearable electronics
  • Subdermally implantable electronics
  • Smart paper
  • Stick with sensors

Our information on the project was mainly based on the presentations provided in the course materials and fast googling. Thus, the ideas were created openly by all members within the group. There were little restrictions on the scope of ideas. We found that it is better to have many ideas although many of them might turn out to be impossible. We also found that it is crucial to have a meeting with the representatives of Canatu, the competences of which we need to analyse. Furthermore, we need a lot more information about Canatu’s interests and goals as well as CNB film itself.

One concept that we found very interesting was a glove with sensors. The glove could be used to wirelessly control electrical devices, such as computers and media systems, but also give signals to other persons. The thumb or any of the fingers of the other hand could be used to control the touch surfaces on fingers. These surfaces could also work as bendable screens to digitally show current functionality. Touch screen bracelet could be used to program the glove and would also contain the transmitter for wireless signal. The picture contains examples of the possible controls.model glove

In the first feedback session we were given some ideas. The most important one was that even though none of us really is expert at building electronic devices, we should explore prototyping possibilities. Perhaps we can find someone with right tools and expertise to manufacture items based on our plans. We should also discuss with our partners at Canatu about the kind of prototype model they need.

Challenges

Our project needs to tackle a couple of practical problems to truly succeed. The most obvious challenge is the lack of expertise of our members; over half of our team has a business or management background. Building a functional prototype requires practical knowledge in electronics which is a skill only one of our members possesses. Another practical issue is time – many of us have grand visions and ideas of various cool concepts but we have to keep in mind the limitations of time and resources. Finally, many of our current ideas require complementing technologies. This creates its own issues due to said technologies’ own limitations and cost.

Next Week

Next week will be the first full week with the project and we’re trying to arrange a meeting with Canatu in the beginning of the week. The first goal of the meeting is to understand the fundamentals of the CNB technology, it’s possible limitations, supplementing technologies (touchscreens) and get feedback on our current ideas. Second goal for the meeting is to discuss the ratio of prototyping and conceptual design and also the possibilities of prototyping resources outside of our team.

After the Canatu meeting we’ll try to come up with more ideas and develop the current ones even further. We might also go through current electrical engineering courses, and see if some of them has practical assignments if this is a relevant opportunity after discussing with Canatu. We’ll also work with the project plan.

Thanks for reading and see you next week!

TEAM PHOTO