Kritika Iyer

This was research done as a continuation of the project done in my deep learning class. I obtained two other datasets; the CAFE data set and the KDEF data set. These had images of higher resolution but less data than the Kaggle data set. After consideration, we decided to consider their facial features instead of the image as a whole and used the Kaggle data set, CAFE, and KDEF and retrained the network. We also classified the emotions into three categories. Positive, negative, and neutral. Doing this gave a final accuracy of 85.19%

We took inputs from behavioral analysts and used those to train on videos of various children to detect attention level. We used different models to compare the results and validate. These ranged from 80 - 94% accuracy. We used only two classifications of on task and off task.

Since this is part of ongoing research, all the data and results are confidential. I cannot show any videos of the children and how the algorithm worked on the children.

Autism Spectrum Disorder(ASD) is a neurological disorder that causes the late development of motor and social skills. PABI(Penguin for Autism Behavioral Intervention) is a robot that helps children with ASD improve their social skills. It is paired with a tablet and gives various audiovisual queues as positive reinforcement for the right answers.

This project furthers the assistance of the robot by making it capable of recognizing emotions. Doing this can show how much of the sessions are helping the child and keeping them involved.

The FERC Kaggle data set, consisting of 7 emotions (anger, disgust, happy, sad, surprise, fear, and neutral) was used for this project. The dateset had some bad images in it which were removed. It also had an uneven amount of data for each label so, anger and disgust were combined into one as they exhibit a similar negative sentiment. Finally, the data was flipped and augmented to have the same number of samples in each label. It was then put into a pickle file and run on the model. The model consists of various Convolutions, Max-pooling, Flattening, Batch-normalizations, and dropout layers with relu and softmax activation functions.

The maximum accuracy reached on these emotions was 57.22%. On seeing the confusion matrix, it was clear that fear was the weak link. After removing fear, the model trained to an accuracy of 68.50%. The highest accuracy recorded on this data set is 71.2%.

The video below shows how the trained model works on live video test data. I tweaked the final layer to show unsure if the confidence was less than 50%.

Some robots have a client-server setup where the client dictates what to do and, the server performs the task. In surgical robots, teleoperation is a well-used structure. The surgeon works on the client-side controller and, the server follows those motions on the patient. In this project, my team and I aimed at making a modular teleoperation framework that could help incorporate any client device into any server device. We did have issues automating the inverse kinematics so, we kept the server device constant(ABB IRB 120). We can vary the server device by entering the inverse kinematics for any device.

In the video shown below, we decided to work with a haptic device as the client to get haptic feedback during surgeries. The haptic feedback is hard to show in the video. In the end, there is a separate segment showing the force applied by the motors against the direction of motion. Actuating the haptic device(geomagic touch) is displayed on Rviz. There is a bumper sensor coded at the end of the ABB arm which sends signals back to the Geomagic touch via ROS. The force was applied to the motors.

Manta rays are supremely efficient swimmers and can withstand a wide range of temperatures and pressures. To mimic their body movement would mean to be able to do deep-sea navigation for extended periods. It would also help study manta ray behavior for oceanographers and marine biologists.

In this project, I modeled a manta ray fin from anatomically correct ratios in Solidworks. Then I analyzed this fin in Comsol. I did deflection analysis, stress analysis, flow analysis with a velocity of 5 m/s for water. It helped me understand which parts of the fin endure more stress and need more reinforcement while designing. The video shows the results of this analysis.

During surgery on organs that move involuntarily, even under anesthesia, the surgeon moves the instruments to compensate for this motion. It means that it is the surgeon's job to synchronize their movements to the patient's. To make their job easier, my team and I worked on a model that would help the surgeon operate as the system takes care of the motion compensation.

We achieved this on a simulation of the Davinci PSM and MTM models on gazebo using ROS. The PSM moved in a combined motion of the MTM and the compensated tissue movement. In the simulation, the platform moves in the form of a randomly generated Fourier transform to simulate tissue. We used an Extended Kalman Filter(EKF) to predict the next position of the platform. The video below shows the results of this model.

There is a boom in interest in self driving cars over the past few years. This project focuses on an algorithm that helps the car stay in lane as well as avoid obstacles safely and return to the lane. This was done using fmincon and yalmip libraries in matlab to optimize the path and produce primitives and reference path lines.

The video below shows the red line (the car) avoiding the blue dot (the obstacle) and rejoining to the keeping the blue stars(the reference trajectory primitives).

Manta rays can swim very efficiently due to the way they flap their fins. In this project, my team and I wanted to try and replicate this motion using smart materials. We designed a manta ray fin mold in Solidworks and then got the mold 3D printed. We filled the manta ray models with silicone ecoflex-30. Some manta rays had Nitinol wires in them during molding that had been pre-trained to fold in one direction, while others had wires put in them after being molded. Another model was made of a mold with chambers for air to go in.

We then applied electricity to the nitinol wires to compare how pre-trained wires worked against the wires we pushed in after molding. We also tested the pneumatic manta ray to see how it would compare to the electric actuation. The results are in the video below.

Autism Spectrum disorder(ASD) not only affects children's social skills but motor skills as well. For my Bachelor's thesis, I designed, fabricated, and programmed a kiosk to help children better their motor skills. Various factors have influenced the design. Autistic children can be sensitive to different stimuli like touch or visual cues such as the texture of balls or LED lights.

I included different skill exercises like turning, pressing, picking up, placing, throwing, rolling, and depth perception. I integrated sensors like IR, Ultrasound, and limit switches to track progress. DC motors actuate different motions. One PIC board and one Arduino board control these devices. Programming stores and tracks the progress of each child. The image below shows the final setup.