In a world where we all feel so “connected”, the people that matter most have no method of remotely interacting with us.  My 1 1/2 year has no twitter, or Facebook account to update.  She doesn’t check into fourSquare every time we go to grandma’s house. Even with mobile phones and Skype video conferencing, there is no direct method of me interacting with her without the assistance and supervision of an adult. So basically, we could only interact with each other when another adult has both the time and is willing to put in the effort to set up an environment for us to communicate.

This got me thinking “how can my daughter and I interact during my work hours?”  The first thing I did was identify the main characteristics of my daughter, back when I first started working on this project:

• My daughter was one year old.
• Her vocal communication ability was in the preliminary stages of development
• Her motor skills where still not fully developed, so complex tasks like using a keyboard or mouse where not possible.
• She needs adult assistance to use our everyday communication tools that available to us (moble and land-line phones, computers, etc).

At this young age, children are extremely dependent on their parents and other adults for just about everything.  However, there are also several activities that come as naturally as breathing and eating to a child. Two of those activities that i observed where exploration and play, many times triggered by a common catalyst of curiosity.

Children gradually explore their environment and toys and interacted with them, mastering the new skills that they learn along the way. Quickly they are able to identify all sorts of toy input locations which triggered various types of output, such as audio and visual feedback.

Many people are quick to suggest that we need to expose children to computers at an early age, I don’t disagree.  However, there is no reason that we need to take a child to a some cold desk, when where they are already in-front of their toy box. Why not instead bring the computer to them, in their own natural environment, and make them available on their own terms. Computers come in various shapes and sizes, and are all around us. In washing machines, toasters, and most of the objects we interact with on a daily basis have them. Just look around you right now, I’m sure you would be surprised on the amount of digital technology that is surrounding you at this moment.

Digital toys, and interactive plush dolls are nothing new. Companies suck as Leap Frog have thrived on making toys which have been getting smarter and smarter. With the introduction of micro-controllers embedded deep inside their soft cuddly exterior, there are endless possibilities.  So that was my answer, I would prototype a interactive based toy solution, that both an adult and child could easily use to communicate with each other.  It is too easy to go into fantasy land with a project like this, so here were my self-imposed rules:

1. The toy must not cost more then $39 retail, since that is my personal breaking point for interactive toys.
2. It must work anywhere, and not be totally dependent on a network connection.
3. The child must need zero guidance to interact with the toy.
4. The toy must promote exploration
5. There must be some learning activities.
6. I must be able to interact with the toy from my office

Putting it all Together

I knew that I would like a toy that enabled “me” to interact with my child via audio or video, however I started seeing that as an obvious solution and not very cost of effective.  Why should parents be forced to buy individual toys for each person that would like to interact with the child?  My daughter has a mother, aunts, grandparents, etc. Would it be feasible of me buying one for each person?  More importantly, would the child be able to interact with such a fragmented family of toys at such an early age? What would be the total cost, $39 a piece? That is pretty steep, and won’t work as a solution in my opinion.  Taking all these questions into account gave me my “ah-ha” moment.  My family and I are part of my daughter’s social network, as exclusive as it may be. Looking at the toy as a social network would mean incorporating several functions that a typical social network would have.  I decided that the most important social network functions that my toy should support where:

• Avatars
• Buddy List
• Messaging

The Birth of my ChatterPillar

I decided to go with a completely modular design, so I could play off the metaphor of a family unit. One unit, one toy, multiple facets. Each facet containing unique actions associated to that family member. The modular design would also allow me to add and remove modules as I please.  This allows my toy to take a very literal approach to a buddy list.  If you wanted to add a buddy you would have to physically add your buddy to the toy and vice-versa. It also allowed for an interesting business model, add all the costly circuitry in the head (the brain) and allow for less expensive body modules to take advantage of the brains powersupply, and various other components like speakers, memory, etc. This along with some inner circuit  communication was all i needed to create the beginnings of what turned out to be myChatterPillar.  Bellow is a video showing my initial alpha prototype of myChatterPillar in action.

Main Features

My ChatterPillar has several features that have been incorporated into the design.

1. There is a photo sleeve on each of the detachable modules which serve as an avatar.
2. There are 3 audio switches. The one beneath the photo is to identify the family member, and the two on the legs is for adding any other audio, like messages, song, etc. These audio buckets are created online by the adult via a accompanying web site, and the files are later synced to the toy via a tethered USB connection.
3. Adults could remotely communicate with the toy and send messages to it via familiar technology such as email, twitter and SMS
4. Embeded assessments to test the child’s familiarity with their family and colors.

Where to go from here

My next steps are to take this toy out of it’s bread board stage, and into a more complete embodiment. I initially looked at this project as a method of communicating with my daughter, but would be great to get it out there some day so other families could also enjoy interacting with their young children. To be continued….

I have spent the past few month deconstructing the cognitive, emotional and social attributes of children under the age of two. I have been particularly interested in understanding why children are more responsive to some toys, then they are for others. What makes a successful toy, and why? What strategies could be used in the development of a toy to foster learning opportunities, along with several other question regarding this ripe age and this medium.

This age is quite interesting since you are not only designing for the child, but also for the parent, who acts as the gatekeeper. This adds an additional level of complexity, and understanding that needs to be understood. So the first question I asked myself was, “What does a parent look for in a toy?”

According to the book, “The Blockbuster Toys”, parents look for toys that;

• Child’s Safety
• Child’s Continued Enjoyment (play value)
• Child’s Creativity
• Child’s Mental & Physical Development
• Child’s Success
• Child’s Health
• Child’s Love

I added some additional bullets that I’ve noticed parents, including myself look into when purchasing a toy:

• Brand Loyalty: Parent could be very loyal to brand names.  There is a strong element of trust when a parent purchases a toy for their child.  Parent trust that Fisher-Price would not put a toy on the shelf that would be dangerous. Others may trust that Leap Frog toys will help their children become smarter.  Your options here are either try to license your toy with a company that has the brand recognition that could help proppel your toy, or make sure to market your company in a way that best reflects the goals of your merchandise.
• Cleaning: How easy is it to clean this toy? At this age kids put toys in their mouths, and parents are sometimes extra cautious constantly cleaning their toys.
• Durability: Will this toy sustain all the punishment that a young child puts on the toy?
• Cost: This one really depends, but is a factor.  I some condition where other bullets such as “Child’s Continued Enjoyment” are high, parent’s may make addition investment. Your goal as a designer, should be to keep the cost as low as possible. I’ve hear that you should imagine all the parts of a toy in a bag and then multiply the cost by 4. That should give you an idea of what the on the shelf cost would be.

What is going on in their little heads?
Although each child is unique, there are some common developmental characteristics that we could use to help and identify common attributes. Through these attribute I’ve been hopping to uncover some rough guidelines to follow when creating toys for this age group.

Albeit, many toys on the market rely on gimmicks or piggy back on the shoulders of licensed characters to attract a child’s curiosity. However, all children have emotional needs that they expect to get fulfilled during play. According to the book The Blockbuster Toy, “The fulfillment of these  deep emotional needs is the fun that creates a smile” , and this results in a child’s consistent play interactions with the toy.

So what are some child’s emotional needs that anyone designing interaction for children in this age group should look into?

• Pride
• Self-Esteem
• Appreciation/Attention
• Accomplishment/Mastery
• Power/Empowerment
• Control
• Silliness
• Independence/Freedom (harmless rebellion)
• Belonging
• Love Nurturing
• Security
• Fear Reduction/Bravery
• Sensory Gratification
• Mental and Physical Development

I narrowed down this list of bullets that fall inline with the child’s development at this point. As they get older, things like fantasy and role playing also start to appear.

What do children know at this age?

The first place to look, is at the child’s stage of development. Jean Piaget, a Swiss epistemologist that focused his research around the development of children of this age to be in their Sensory Motor Stage of development:

Sensorimotor stage: from birth to age 2. Children experience the world through movement and senses (use five senses to explore the world). During the sensorimotor stage children are extremely egocentric, meaning they cannot perceive the world from others’ viewpoints. The sensorimotor stage is divided into six substages: “(1) simple reflexes; (2) first habits and primary circular reactions; (3) secondary circular reactions; (4) coordination of secondary circular reactions; (5) tertiary circular reactions, novelty, and curiosity; and (6) internalization of schemes.”

Love and Security

Children at this stage of development need to feel safe and loved. Many of the children in the lower part of this demographic still rely solely on their mother for nourishment.  This bonding along with all the hugs and kisses they get from their parents, help a child feel the security they need to find their place in this world, and with their family. Children will sometimes show their love and nurturing to a toy doll by snuggling up with it.

Language

Children at this age recognize voices and the emotions that they convey.  Although a child at this age is just beginning to create a vocabulary, they do understand much of what you are saying to them.  Like little sponges they start to develop a schema for spoken words and actions and objects. Eventually, towards the child starts to experiment with their own voice to mimic sounds.

Babies of this age could also identify familiar voices.  If you put mommy on speaker phone they may start looking around the room to see where mommy is.

This is a perfect time to give a child an opportunity to make connection and experiment with language to help with their development.  The more a toy allows a child to be vocal, the quicker they will learn how to control their vocal instrument.

People and Objects

Children at this age are good with faces. Babies could recognize face structures and they use physical characteristics,  characteristic expressions, postures and movements to identify specific individuals to create mental models of people. Many times, when a baby sees a photo of mommy, they could easily make the correlation between mommy and the photo, and results in a smile and excitement. This same process happens with objects that they discover and interact with, however these mental models are very generalized. Although they may know what a doggie is, they may also think a cow is doggie, or any other animal that has 4 legs.

How do children play at this age?
Children at this preliminary stage of development are self-centered. They don’t do well in collaborative play since their social development has not ripened. Concepts like “sharing” do not come naturally till later in this stage, or possibly well into the following stage, depending on how quickly their socialization skills develop. Through these formative years, you will see more “parallel play”. Typically, children are in their own little world playing, even if there is another child playing right next to them. Creating a toy that requires collaboration or cooperation would probably be too advanced for this age group.

Structure is a “no go” with young children. They are more interested in “free play”. These children lack the cognitive ability to play a game with rules or activities that require advanced forms of interaction. Keeping it simple with clear feedback is the way to go. Simple switches with blinking lights and audio work well, but don’t feel as though that is all you could do. You could get creative and create some interesting simple interaction from a ball with an embedded tilt sensor.  As with any design limitation, it could usually be solved with creativity.

This age group is packed with little explorers, constantly scanning the room and objects for something to interact with.  They enjoy to the feeling of empowerment when their actions result in sort of feedback (lights, audio, parent’s laughter).  Once they discover a “cause and effect” interaction, they will sometimes continue testing their hypothesis over and over.  If you have ever seen a child that discovers how to open a kitchen cabinet, you will know what i mean. Open, close, open, close … parent buys a cabinet lock.
Conclusion
After following all these guidelines the truth of the matter is that the toy industry is a tough to break into.  There are only a handful of successful toys that make it big each year, while thousands of others collect dust on shelves, if they get on the shelves at all.  The one piece of advice that I read though my research which really made sense is, “Do it for the smiles, not the money”.  That is the essence of toy building, before companies like Mattel and Hasbro came along, parent would create toys simply for getting the satisfaction of watching the smiles on their children’s faces. The other piece of advice is don’t get married to any idea. If you get too emotionally invested in an idea, it will not allow you to see it’s flaws. Unless you are making the toy for yourself, then you need to step back and see what of your audience’s goals you have accomplished.

References
What Kids Buy and Why: The Psychology of Marketing to Kids
Blockbuster Toy!, The: How to Invent the Next BIG Thing
Marketing to the New Super Consumer: Mom & Kid

From Arduino Site:

Wire.begin()
Wire.begin(address)
====
Description
Initiate the Wire library and join the I2C bus as a master or slave.
====
Parameters
address: the 7-bit slave address (optional); if not specified, join the bus as a master.
====
Returns
none

I interpreted this explanation as Wire.begin(SomeAddress) would default as a slave, and not be allowed to be a Master. Being as curious, and stubborn as I typically am, i decided to forget what I thought I was being told, and instead try to break the library, so I could then see where it broke, and attempt to adjust it in a way that better suited my need. Surprise, surprise, it worked, and not need to change anything at all! The address parameter is optional and defaults as a master role if no address is set. However if you do plan on sending messages to the master, you could switch speaker and listener roles via a slaves Wire.onReceive(handler) rather then polling using onRequest(handler).

I then decided to do a search on google for just “i2c”. When I clicked on the Wikipedia entry it all made sense in the first sentence of the entry summary:

I²C (Inter-Integrated Circuit) (pronounced /ˈaɪ skwɛərd ˈsiː/ or /ˈaɪ tuː ˈsiː/) is a multi-master serial single-ended computer bus…

This basically states that no one microcontroller (unless you do not specify an address) could claim being a master at all time. They could be bumped off their soupbox by another microcontrol and assume the role of listener/slave at anytime.

I decided to take a trip to my daughter’s toy collection to examine some of the sensing capabilities that her electronic toys offer. I will specifically focus on the toys input capabilities:

(At the end of the post there is a video of me trying to figure out how the LeapFrog Tag Jr works. Let me know your thoughts.)

Fisher-Price Elmo Live Encore

This is by far one my daughters favorite toys. The toy is modeled after Elmo, a Sesame Street character, which most children a quickly drawn to by his bright red color, large eyes, and youthfully fun voice.

Inputs

• Push Buttons: Typically buttons on toys are visually obvious, however Fisher Price decided to create an exploratory interaction by spreading out four button throughout the plush doll. Three of the buttons are hidden under the fabric, on the toe, chest, and back, and the final button switch is closed by squeezing his nose.
• Tilt Sensor: One of the nice additions that the creators added to the toy was integration of a Tilt Sensor. When Elmo falls on his side, he politely ask for help to stand backup, which is sometime accompanied with a joke.

LeapFrog Learn & Groove™ Musical Table

We originally got this toy when our daughter was showing signs of wanting to stand on her own. The toy itself is stuffed with several auditory and visual feedback that really grabs her attention.

Inputs

• Push Buttons: Typically buttons on toys are visually obvious, however Fisher Price decided to create an exploratory interaction by spreading out four button throughout the plush doll. Three of the buttons are hidden under the fabric, on the toe, chest, and back, and the final button switch is closed by squeezing his nose.
• Toggling Buttons: From watching my daughter play with this section of the table, I noticed that the design decision was probably based around learning not only shapes, but also cause and effect. When she presses one, the previously pressed button switch pops up to an open state.
• Flipping : This might be one of my daughter favorite interactions. It may be partly due to the fact that we read a lot of books to her, or it might just be an intrinsic curiosity that makes her want to flip things over. Feedback is dispatched when the child flips the pages of the book on the center of the toy, and also when they open the small compartment on one of the sides. I would imagine that the switch is hidden on the hinge of the book. Each page has individual hinges that are associate with it. Once the page is flipped and crosses the angular threshold an embedded audio file is dispatched, along with some dancing LEDs.
• Slider: The slider, is a “slider” in the physical interaction sense of the word. However the slider does not send an analog signal, or at least is doesn’t technically need too. I believe that the reason for the design decision was mainly an educational one to create opportunity to exercise multiple sensory motor functions (sliding, twisting, pushing, flipping, etc.)

LeapFrog My Pal Violet

This product is actually one of my favorites. In the exterior it seems like a pretty straight forward digitally enhanced plush toy, but it add another layer of interaction for the parent, which evolves connecting the toy to the computer via USB and downloading audio files onto the toy. There is also some slight visual feedback from the LED that is housed in the plastic bone around the toy’s neck.

Inputs

Push Buttons: There are 4 normally open push buttons, each living on one of the plush dog’s paws. All the buttons, have iconic representations of the type of audio that it will dispatch once the switch is closed: Music, Action, Lullaby , and Power Off.

Parental Interaction
Keyboard/Mouse: The toy comes with a desktop application that syncs with the toy via USB. The parent could then select which audio files to have the available to the toy. They are also able to personalizing some the interaction, by selecting from a list of names.

LeapFrog Spin and Sing Alphabet Zoo

This was probably the first toy that my daughter mastered at a couple months old. She almost looked like a boxer spinning the all the letters round and round, and enjoy the audio that is dispatched on the toys active state. Once the action stops, and the spinning section comes to a stop, the audio dispatches which animal or letter is in between the arrows, sort of like the large wheel from Wheel of fortune.

Inputs

Without gutting this toy (my wife would kill me) I’m not too sure how they made this work, however my best guess is:

Metal roller/ball bearing & Metal Plate contact: each of the 25 selections have individual plates that are connected to some sort of Shift register or multiplexer. As the the spinning action occurs the fixed conductive roller stays in contact with each of the plates that passes by. Once an individual contact is closed for about a second, the spin cycle is assumed to have finished, and the appropriate audio file is dispatched. Not sure if this is accurate since this may add friction. I can’t be sure unless i open it up.

Here is a video of my daughter playing with this toy:

LeapFrog Learn & Groove Counting Maracas

These are bionic baby rattles/maraca that sense when they are being shaken. Once they are shaken an array of lights and melodies start to play. These are lots of fun, and has an added affordance of making adults act like complete fools.

Input

• Slide Switch: This is just a simple multi state switch to adjust various mode settings
• Accelerometer or Tilt Switch: Not sure exactly how they are sensing movement, but typically you would use either a tilt switch, or accelerometer. I would put money on the tilt switch mainly for cost, and being that the accelerometer may be overkill.

Fisher-Price Go Baby Go! Crawl-Along Musical Ball

We bought this toy when the baby was in the brink of starting of crawling. The interaction involves rolling the ball and watch the illusion of the monkey keeping it’s balance, while letting out some humorous squeals.

Input
The only input for this toy is sensing that the ball is in movement. Since the ball roll only in one direction, I would imagine a spinning tilt switch is all that was needed to sense when the ball was in motion.

LeapFrog Tag Junior Book Pal – Purple

One of my friends bought this toy for my daughter, which is a bit advanced for her, however extremely impressive. This toy serves a scaffold for children that are learning to read. By simply touching the pen on the area of the word, it dispatches an audio files that speaks the word that they child selected.

Input

It is totally beyond me how this technology works, but I will post up a video of me trying to guess. No idea if will actually be right, and unfortunately I need to wait till my daughter could read before I could take it apart, so a guess will do for now.


My guess on how the LeapFrog Tag Junior works:

Update: So it turns out I was correct…woohoo! I did a few searches and ran into the LeapFrog FlyPaper Pen. I didn’t event know LeapFrog made this. Anyhow, I downloaded the manual and this is what it says under “How it works”. (would have been a lot easier if they added that section in the Tag manual).

I also found another guy who made a video about the same exact topic, and had the same answer. http://www.youtube.com/watch?v=A6yKZ1wjUgg

In the accompanying video, I give a quick run through of how the circuit works. Essentially, the main goal that I was able to achieve was that I would be able to clone one the inner circuits and add it right in the middle. All I would need to do is connect them to the 2 digital pins, 2 analog pins, one ground, and one 5V series, and it should work without any additional rewiring (although the idea is to make the sub circuits slightly different in their interaction, yet the same in their communication), asides from programing the EEPROM (I used the 24LC256 from SparkFun for $1.95) to contain a unique identifier. The secret to success…diodes..lots of them.

I based my code on a great snippet of code to help me get the I2C communication working on my EEPROMs. I also checked out a helpful tutorial which also helped me out substantially.

One quick tip: Keep in mind that Arduino has an EEPROM mounted on the chip. I intially made the silly mistake of using the EEPROM code that is found on the Arduino tutorial page. This code was reading and writing data on my Arduino’s board mounted EEPROM, which i quickly realized once I realized that the unique data I was writing on each external EEPROM chip did not seem to be working properly.

So why go through all this extra effort? Basically there are two reasons.

1) allowing a person to customize and interchange their toy gives them an added sense of ownership, and taps into an emotional type of attachment.
2) it is pretty easy to come up with a ways of taking the ideas behind this circuit, and give very young children the ability to construct abstract concepts such as flow. This in turn makes this simple interaction material a digital manipulative with extraordinary educational benefit.

One of my points of inspiration was MIT’s Mitchel Resnic’s Programmable Beads , and other work with digital manipulative that I’ve spoken about on other posts.

Onto phase 2…

Here is a video of Mitch Resnick speaking about his Lifelong Kindergarten group at the MIT Media Lab

Bellow is an updated version of my previous Arduino coding. I plan on switching the status updating to the Processing side, instead of the Arduino side. If this where to be a real product, it would be easier to download a software update, then to have to reprogram the toy itself. I will add that to my todo list, once i get all the major functionality working.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96

// BUTTONS
const int buttonGreen = 8;
const int buttonRed = 9;
const int buttonYellow = 10;
 
//LEDS
const int ledGreen =  2;
const int ledRed =  3;
const int ledYellow =  4;
 
//STORE BUTTON STATES
int greenState = 0;
int redState = 0;
int yellowState = 0;
 
int lastActiveLED = 0; // Keep track of the activeLED
 
void setup() {
  Serial.begin(9600);
 
  pinMode(ledGreen, OUTPUT);
  pinMode(ledRed, OUTPUT);
  pinMode(ledYellow, OUTPUT);
 
  pinMode(buttonGreen, INPUT); 
  pinMode(buttonRed, INPUT); 
  pinMode(buttonYellow, INPUT); 
 
}
 
void loop(){
  // CHECK BTN STATES
  greenState = digitalRead(buttonGreen);
  redState = digitalRead(buttonRed);
  yellowState = digitalRead(buttonYellow);
 
  /**********
   *  Each of the following conditionals only sends one Serial message
   *  per button press
   **********/
 
  //GREEN
  if (greenState == HIGH) {
   //check if last active LED was not Green 
    if(lastActiveLED != ledGreen){
      //print the respective message for each button only once per mouse press. End message transmission with '10' which is linefeed.
      Serial.print("green");
      Serial.print(10, BYTE);
    }   
    //store green as last active press
    lastActiveLED = ledGreen;
    digitalWrite(ledGreen, HIGH);
  } 
  else {
    //clear out last active so you could reclick it
    if(lastActiveLED == ledGreen){
      lastActiveLED = 0;
    }
    digitalWrite(ledGreen, LOW);
    delay(100);
  }
 
  //RED
  if (redState == HIGH) {    
    if(lastActiveLED != ledRed){
      Serial.print("red");
      Serial.print(10, BYTE);
    }   
    lastActiveLED = ledRed;
    digitalWrite(ledRed, HIGH);
    delay(100);
  } 
  else {
    if(lastActiveLED == ledRed){
      lastActiveLED = 0;
    }
    digitalWrite(ledRed, LOW); 
  }
 
  //YELLOW
  if (yellowState == HIGH) {    
    if(lastActiveLED != ledYellow){
      Serial.print("yellow");
      Serial.print(10, BYTE);
    }   
    lastActiveLED = ledYellow;
    digitalWrite(ledYellow, HIGH);
    delay(100);
  } 
  else {
    if(lastActiveLED == ledYellow){
      lastActiveLED = 0;
    }
    digitalWrite(ledYellow, LOW);
  }
}

Now that we are sending message, we need to listen. I decided to go with Processing since I’ve always been curious about it, and it is about as seamless of a coding experience as you could possibly get with Arduino.

In order to get the Twitter4J Java Library working you first needed to drag the .jar file directly onto your Processing sketch, and that is pretty much it. Bellow is the Processing code that simply listens for a Serial Event (Arduino button press), and then sends the status message over to Twitter.

Being a total N00bie to Processing, one additional tip that confused me was Fonts. Processing doesn’t allow True Type Fonts, so if you want to add text to your project you need to go to:

Tools > Create Fonts…

and then select the font you would like to use. The Processing IDE then converts the font to a .vlw file. You could find the file in a folder named ‘data’ that gets created within your projected folder.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

/*************
*    based on:  http://processing.org/reference/libraries/serial/serialEvent_.html
*************/
 
import processing.serial.*; 
 
Serial myPort;    // The serial port 
PFont myFont;     // The display font 
String inString;  // Input string from serial port 
int lf = 10;      // ASCII linefeed 
 
Twitter twitter;  // Twitter
 
//Going to get oAuth working instead of this, but this will do for now
String username = "YOUR-TWITTER-USERNAME"; // you Twitter Username Here
String password = "YOUR-TWITTER-PASSWORD"; // your Twitter Password Here
 
void setup() { 
  size(400,200);
  twitter = new Twitter(username,password);
  myFont = loadFont("AppleGothic-48.vlw"); 
  textFont(myFont, 18); 
  println(Serial.list()); 
  myPort = new Serial(this, Serial.list()[0], 9600); 
  myPort.bufferUntil(lf); //wiat for line feed to specify end of serial buffer
} 
 
void draw() { 
  background(100); 
  text("color selected: " + inString, 10,50); 
} 
 
void serialEvent(Serial p) {
  inString = p.readString();//read serial string
  //For some reason this only wanted to work in a try catch
  try
  {
    Status status1 = twitter.updateStatus("Arduino's favorite color is "+inString);//update twitter status
  }
  catch( TwitterException e)  {
    println(e.getStatusCode());
  }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

// BUTTONS
const int buttonGreen = 8;
 
//LEDS
const int ledGreen =  2;
 
//STORE BUTTON STATES
int greenState = 0;
 
 
void setup() {
  Serial.begin(9600);
 
  pinMode(ledGreen, OUTPUT);
  pinMode(buttonGreen, INPUT); 
}
 
void loop(){
  // CHECK BTN STATES
  greenState = digitalRead(buttonGreen);
 
  //GREEN
  if (greenState == HIGH) {
  Serial.print(ledGreen, DEC);
    digitalWrite(ledGreen, HIGH);
  } 
  else {
    digitalWrite(ledGreen, LOW); 
  }

When i check my serial monitor I receive something like:
22222222222222222222222222…..

That won’t work, since I plan for each of the Serial events to make an API call to Twitter. This activity would wipe out my allotted request in a matter of a couple of clicks, and probably would result in getting my account suspended due to unusual activity. So bellow is a quick fix which store individual button clicks, and prints out the Serial output once. I also decided against using any delays, since that may have potentially ended up in ignoring a button press.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

// BUTTONS
const int buttonGreen = 8;
const int buttonRed = 9;
const int buttonYellow = 10;
 
//LEDS
const int ledGreen =  2;
const int ledRed =  3;
const int ledYellow =  4;
 
//STORE BUTTON STATES
int greenState = 0;
int redState = 0;
int yellowState = 0;
 
 
int lastActiveLED = 0; // Keep track of the activeLED
 
void setup() {
  Serial.begin(9600);
 
  pinMode(ledGreen, OUTPUT);
  pinMode(ledRed, OUTPUT);
  pinMode(ledYellow, OUTPUT);
 
  pinMode(buttonGreen, INPUT); 
  pinMode(buttonRed, INPUT); 
  pinMode(buttonYellow, INPUT); 
 
}
 
void loop(){
  // CHECK BTN STATES
  greenState = digitalRead(buttonGreen);
  redState = digitalRead(buttonRed);
  yellowState = digitalRead(buttonYellow);
 
  /**********
   *  Each of the following conditionals only sends one Serial message
   *  per button press
   **********/
 
  //GREEN
  if (greenState == HIGH) {
   //check if last active LED was not Green 
    if(lastActiveLED != ledGreen){
      //print Green since it is a new press
      Serial.print(ledGreen, DEC);
    }   
    //store green as last active press
    lastActiveLED = ledGreen;
    digitalWrite(ledGreen, HIGH);
  } 
  else {
    //clear out last active so you could reclick it
    if(lastActiveLED == ledGreen){
      lastActiveLED = 0;
    }
    digitalWrite(ledGreen, LOW); 
  }
 
  //RED
  if (redState == HIGH) {    
    if(lastActiveLED != ledRed){
      Serial.print(ledRed, DEC);
    }   
    lastActiveLED = ledRed;
    digitalWrite(ledRed, HIGH);
    delay(100);
  } 
  else {
    if(lastActiveLED == ledRed){
      lastActiveLED = 0;
    }
    digitalWrite(ledRed, LOW); 
  }
 
  //YELLOW
  if (yellowState == HIGH) {    
    if(lastActiveLED != ledYellow){
      Serial.print(ledYellow, DEC);
    }   
    lastActiveLED = ledYellow;
    digitalWrite(ledYellow, HIGH);
    delay(100);
  } 
  else {
    if(lastActiveLED == ledYellow){
      lastActiveLED = 0;
    }
    digitalWrite(ledYellow, LOW);
  }
}

If anyone knows of any other methods of doing this, let me know!

I tried to keep both programming and circuitry as slim as I could, and I think that worked out pretty well. Not sure if i will add anything else to this, but if anyone has any ideas, asides from my coworker’s recent tweet, then Iet me know.

This source code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220

 /*
   Simon Sez
 
 
  created Feb 10 2010
  by Alex Britez
 
  http://blog.unthinkmedia.com/?p=475
 
  */
 
#include <Tone.h>
 
Tone audioFeedback;
 
//DEFINE MOTOR
int transistorPin = 12; 
 
int speakerPin = 6; 
 
//DEFINE LEDS
int ledWhite = 11;
int ledGreen = 10;
int ledRed = 9;
int ledYellow = 8;
 
char leds[] = {
  ledWhite,ledGreen,ledRed,ledYellow};
 
//DEFINE SWITCHES 
int switchWhite = 5;
int switchGreen = 4;
int switchRed = 3;
int switchYellow = 2;
 
//AUDIO
int notes[] = {
  NOTE_D4,NOTE_E4,NOTE_F4,NOTE_G4};
//this plays if you win
int winNotes[] = {
  NOTE_D4, NOTE_E4,NOTE_C4,NOTE_C3,NOTE_G3 };
int durations[] = {
  500, 500, 500, 500, 1000};
 
 
 
//
int playbackCount = 0;//autoplay selection count
int selectionCount= 0;//player selection count
 
int currentLevel = 0;//current game level
 
boolean watchMode = true; //Simon's turn, false means it is the players turn 
 
//store which switch is currently active?
int pressAry[] = {
  0,0,0,0};
 
//Storing the squence that gets randomly generated, Num in brackets states how long sequence is [#]
int sequenceAry[7];
int sequenceAryLegnth = sizeof(sequenceAry) / sizeof(int);
 
void setup()
{
  pinMode(ledWhite, OUTPUT);
  pinMode(ledGreen, OUTPUT);
  pinMode(ledRed, OUTPUT);     
  pinMode(ledYellow, OUTPUT);
  pinMode(switchWhite, INPUT);
  pinMode(switchGreen, INPUT);
  pinMode(switchRed, INPUT);     
  pinMode(switchYellow, INPUT);
  pinMode(transistorPin, OUTPUT);
 
  randomSeed(analogRead(0));//Add this so sequence is truely random
  audioFeedback.begin(speakerPin);
  createRandomSequence();
}
 
void loop()
{
  /************
   *  Check to see if simon/player mode
   *************/
  if(watchMode == true){
   /************
   *  SIMON SEZ: leds automatically blink in random sequence
   *************/
    //have you won?
    if(playbackCount <= sequenceAryLegnth){
      //has simon finshed showing current sequence?
      if(playbackCount < currentLevel){
        //Blink proper LED from leds array
        int activeLED_index = sequenceAry[playbackCount];
        digitalWrite(leds[activeLED_index], HIGH);
        //Play corrisponding note
        audioFeedback.play(notes[activeLED_index]);
        delay(200);
        digitalWrite(leds[activeLED_index], LOW);
        audioFeedback.stop();
        delay(200);
        playbackCount++;
      }
      else{
        //simon is done, now it is players turn
        watchMode = false;
      }
    }
    else{
       /************
       *  PLAYER WINS!
       *************/
      playerWins();
    }
  }
  else{
 
    //Read States of all Switches
    int whiteState = digitalRead(switchWhite);
    int greenState = digitalRead(switchGreen);
    int redState = digitalRead(switchRed);
    int yellowState = digitalRead(switchYellow);
 
    int statesAry[] = {
      whiteState,greenState,redState,yellowState};
 
    //loop across all 4 switch states
    for(int x=0; x<4; x++){
      int switchState = statesAry[x];
      if(switchState == 1){
        audioFeedback.play(notes[x]);
        digitalWrite(leds[x], HIGH);
        delay(200);
        audioFeedback.stop();
        digitalWrite(leds[x], LOW);
 
        if(pressAry[x]==0){
          if(sequenceAry[selectionCount]==x){
            selectionCount++;
            pressAry[x] = 1;
          }
          else{
            resetGame();
          }
 
        }
 
      }
      else{
        pressAry[x] = 0;
        // turn LED off:
        digitalWrite(leds[x], LOW); 
      }
 
      //Check to see if we could incrament level
      if(selectionCount == currentLevel){
        advanceLevel();
      }
    }
 
  }
 
}
 
 
void createRandomSequence(){
   for(int x=0; x<sequenceAryLegnth; x++){
      sequenceAry[x] = random(0, 4);
   } 
 
}
 
/************
 *  Play winning melody
 *************/
void playerWins(){
  for (int thisNote = 0; thisNote < 5; thisNote ++) {
    // play the next note:
    audioFeedback.play(winNotes[thisNote]);
    // hold the note:
    delay(durations[thisNote]);
    // stop for the next note:
    audioFeedback.stop();
  }
  // hold before repeating:
  delay(3000);
}
 
/************
 *  Incrament player one level
 *************/
void advanceLevel(){
  resetValues();
  currentLevel++;
  delay(200);
} 
 
/************
 *  Player pressed wrong button
 *************/
void resetGame(){
  //DISPATCH HAPTIC FEEDBACK 
  digitalWrite(transistorPin, HIGH);
  delay(1000);
  digitalWrite(transistorPin, LOW);
  delay(1000);
 
  resetValues();
  currentLevel = 0;
  delay(1000);
}
 
/************
 *  Reset all values so simon knows to start from the first LED in the sequenceAry
 *************/
void resetValues(){
  selectionCount = 0;
  playbackCount = 0;
  watchMode = true;
}

Anyhow, armed with some basic knowledge of electronics, and some programming knowledge, why not! Bellow is my stab at Simon, I even added some haptic feedback just because. The audio is a bit low because it was 1am and I didn’t want to wake up my wife and daughter, but you get the idea.

Before I post up the code, I need to clean it up a little bit and get a bit more acquainted with C. The programing language that Ardiuno uses is pretty bare bones, so a lot of common functions that i have taken for granted don’t exist, go figure. Not a big deal, it will just take some research and tinkering to figure it out.

Another issue i had was regarding resistor setup, an 8 ohm speaker, and it’s affect to what i believe is the audio pitch. The sound that plays during Simon’s turn and the sound of the same note that play during the button press is slightly different. May need to bring out the multi-meter and see how many ohms are being passed through, in both scenarios. I am using the Arduino Tone Library. I’ll add my findings in my next post once i find some free time to get my debugging hat on. Other then that, my first real project went pretty smoothly.

As I mentioned, once I iron out these small bugs, I will post all the source and wiring, then take it apart and move onto my next project. Homework has never been so much fun…ha!

Breaking free of the monitor, and starting my voyage into the physical realms of physical computing, I decided to step-back and analyze some of the physical objects that I regularly interact with. It wasn’t long before I noticed the perfect subject.

The Metro Card Vending Machine is a common place for many New Yorkers. As a matter of fact, I thought it had a rather intuitive design, until this morning. They have incorporated a sexy touch screen design, with large buttons so clumsy fingers don’t press the wrong thing. In my opinion the interface is quite intuitive and rather pleasant experience, or as pleasant as refilling your Metro Card could possibly be.

This morning, on the machine next to the one that i was using, a tourist proved me wrong. It turns out that the touchscreen is just one of several interfaces on the vending machine, and although the tourist easily navigated the metro card touchscreen buying experience, she still needed to interact with other parts of the machine that obviously didn’t receive as much love.

As the tourist selected her card type, amount, and payment type everything seemed great. She then dipped her credit card into the credit card slot, entered in her pin or zip depending on payment type and BOOM! The screen asked her to press “Enter”. Since I was there observing her interaction with the machine, I let her argue with the machine a little while to see how long it would take her to figure it out… At least a minute passed and she started getting visible upset, her friend yelled out from the other side of the turnstile, “Just forget it and go to the booth!” Finally, I stepped in before she became completely ballistic and let her know that the enter is actually on the credit card keypad labeled ENT over the pound sign (#). She thanked me and went on her way.

At that moment the tourist may have felt stupid, but is she really stupid? Could the design be flawed? Stepping back and examining the interface which i have now identified as the “entire machine”, and not just the sexy touch screen, a couple things stand out. The first thing I noticed was that there are two separate inputs, the multi-touch screen and credit card keypad. This inherently splits the users attention jumping back and forth from screen to keypad, possibly causing confusion on what they should be looking at next. Although, moving the keypad to a digital version on the touch screen may seem like a viable answer, this would causes an added level of confusion by interrupting the spatial congruity that the credit card slot and the keypad have. So what is the answer? In the book, The Art of Interactive Design: A Euphonious and Illuminating Guide to Building Successful Software, Chris Crawford describes interactivity as:

interaction: a cyclic process in which two actors alternately listen, think, and speak.

As the tourist was interacting with the Metro Card vending machine she was essentially communicating with the physical object.

“Please enter your zipcode and press Enter”

Obviously, the tourist can’t speak to the machine vocally, but she could communicate her selection through the buttons that are presented. Since the tourist could not find the Enter key she essentially was stripped of the ability to communicate, placing a muzzle on her interactive experience.

A possible solutions that may have worked better by simply rewording the communication, may have looked like this:

“Please enter your zipcode and press # on the keypad bellow”

If you aren’t going to write out ‘ENTER’ and simply use ‘ENT’ on the pound key (#) that now serves a multiple function, then why bother having an enter key at all. Either add a physical key that serves the singular purpose of being ‘ENTER’, or make sure to properly communicate the design flaw to the user to avoid any confusion.

This brought up a couple great points that designers of physical computing should be aware of:

• Don’t assume a task is obvious, although it may be obvious to you it may be confusing to someone else.
• Sometimes you may be forced to use components that are not ideal, but meet the need in other ways such as budget, time and convenience. Do your best to make sure that any confusing interaction is properly scaffolded to minimize error and avoid frustration