On a usual day at office, if an employee wishes to book a meeting room, he or she first has to open their outlook account, check the calendar for its availability and then send a booking invitation. If they directly go to a meeting room, they have no way of knowing whether it is booked or not without checking the calendar.

Smart Meeting Room is a step forward in automating the availability and booking procedure of a meeting room.

The main idea behind this app is to avoid the hassle of opening outlook, checking for room availability and then booking the room in which a user is already in. The user will just have to give a voice command to check the availability of room and another one to book the room. The application will recognize the user’s command and give a speech output. In order to confirm the identity of person, a camera is initiated, capturing the user’s image and recognizing it using the face detection API.

To fully automate a meeting room, an AC control module and a TV control module have been integrated into the app which allows the user to control the room’s air conditioning unit and the TV unit through simple voice commands.

Solution Concept

The application targets controlling the meeting rooms of the company using a simple natural interface. The solution is divided into four modules.

• Checking Room Availability - A user can directly go to a meeting room and ask for its availability using simple voice commands. The application will recognize the user’s command and give a speech output.
• Room Booking - If the room is available, we initiate a camera, capture the user’s image and recognize it using the face detection API in order to confirm the identity of person. Once the user is recognised, a meeting request is sent on behalf of the user and the room gets booked.
• Controlling the AC- The air conditioning unit of the room is controlled using simple voice commands. The application sends the user’s commands to a device attached to the AC which then transmits appropriate Infrared signals.
• Controlling the TV - Controlling the television unit of the room using voice commands works much the same way as the AC.

Benefits

The Smart Meeting Room project eases the way in which an employee can interact with a meeting room.

• Using the Speech Recognition and Face Recognition APIs, it provides for very rich and natural interaction between the user and the application.
• It provides a single interface through which a user can control all the aspects of a meeting room. Whether it is checking for room availability and its booking or controlling the room’s AC and TV, everything can be done using simple voice commands.
• It mitigates the need for finding and using multiple remote controls for the AC and the TV. A simple voice command does it all.
• It removes the hassle of manually checking the calendar on outlook and then creating a booking.

Design and Architecture

The architecture of the Smart Meeting Room system was designed to provide a stable foundation, both for current functionality and for future requirements.

• Universal Windows Application: Universal Windows Platform (UWP) provides a common app platform available on every device that runs Windows 10. The Universal windows app written for the Smart Meeting Room will run on Raspberry Pi 3 installed with windows IoT Core operating system. UWP provides a large set of APIs to work with which makes it easy to integrate richer experiences with your devices such as natural user interfaces, searching, online storage and cloud based services.

  • User Interface: A user interface created using XAML provides users to interact with the system directly.
  • Media Control: The Media Control component in the universal apps enables us to use the webcam attached with our system and show a live camera feed to capture the image.
  • Bluetooth Low Energy (BLE): The power-efficiency of Bluetooth with low energy functionality makes it perfect for devices that run for long periods on limited power sources. We use the Bluetooth GATT profile to connect with the Arduino board in order to send the commands for the AC and the TV operations.

• Microsoft Speech Platform: The Microsoft Speech Platform provides a comprehensive Speech Platform Runtime management in voice-enabled applications. It provides the ability to recognize spoken words (speech recognition) and to generate synthesized speech (text-to-speech or TTS) to enhance users’ interaction.

• Microsoft Exchange Web Service: The Microsoft’s Exchange Web Service allows us to connect to the Office 365 account and access the public folders and calendars and retrieve the free/busy status and availability of the meeting rooms.

• Microsoft Face API: The Project Oxford’s Face API for windows universal apps allows for programmatic access to the cognitive service subscription. We use the API to detect faces in the picture captured, identify the faces and retrieve their stored information.

• Microsoft Office 365 Web API: The Office 365 Managed APIs provides a managed interface for developing the application that allow programmatic access to public folders and calendars.

• Microsoft Project Oxford Face Detection API: The Microsoft cognitive services Face API Detect one or more human faces in an image, organize people into groups according to visual similarity, and identify previously tagged people in images.

• Arduino Application: The application to control the AC and the TV is run on a single board microcontroller such as Arduino Uno.

• Smart Meeting Room Web Application: A separate web application running on Microsoft Azure Platform is made for users to register themselves.

Constraints and Challenges

• The Arduino application is written for a specific model of air conditioner and television. Smart Meeting Room application will not work for any AC or TV.
• We had to manually retrieve the IR commands to be sent using the IR receiver sensor. Every permutation of the AC operation was generated manually, which was a very tedious work.
• While writing the Arduino application, we constantly had to keep in mind the size of the code. Due to limited memory, the number of Serial. Write() were to be kept in check which made logging and debugging a challenge.
• We had to use the Exchange Web Service, writing soap requests and parsing soap responses manually. This was due to the fact that the EWS Managed API generated a popup which cannot be handled in Raspberry Pi.
• We have implemented the Speech Interpretation in low confidence. This might create a problem in proper interpretation of commands due to different voice accents.
• We generated the Bluetooth GATT profile on our own with its services and characteristics exposed. No particular standard was followed.
• For the IR emitter sensor to properly work, it was to be positioned in such a way that it was in a direct line of sight of both the AC and the TV. This became a challenge depending on the how and where the AC and the TV were installed with respect to each other.

Conclusion

The Smart Meeting Room application is a good experiment, it can be extended to implement more features in future such as

• Implementing meeting check in functionality on Smart Display
• Controlling room condition of specific meeting room using a web based dashboard
• Controlling AC and TV of specific meeting room remotely using a web dashboard
• Turning the air conditioner on/off by sensing the presence of person

References

• Microsoft Cognitive Services Face API
• Microsoft Azure Blob Storage
• Universal Windows Platform Speech Recognition
• https://msdn.microsoft.com/en-us/windows/uwp/input-and-devices/speech-interactions
• https://blogs.msdn.microsoft.com/cdndevs/2015/05/25/universal-windows-platform-speech-recognition-part-2/
• Universal Windows Platform Bluetooth GATT
• https://developer.microsoft.com/en-us/windows/iot/samples/blegatt2
• Microsoft Exchange Web Service
• Office 365 OAuth
• https://github.com/OfficeDev/O365-WebApp-OAuthController

This concept was supported by my colleagues Ricktam K and Harsh Godha at IoT Center of Excellence of Nagarro