I’m using the Haply Unity package to test ways to integrate the Inverse3 directly with game engine physics and I’m having difficulties to enable the Bluetooth pen. I’m using an USB Bluetooth adapter in my desktop.
The pen is paired with the computer, but I can’t find it in the Haply windows dashboard app.
I’ve tried connecting it through the unity package too, but I didn’t understand how to use the legacy handle script.
What is the correct procedure to connect the pen inside the unity package?
Hi William,
The Bluetooth connection is sometimes sticky, and the pen needs to be restarted to make a new connection. We’ve add some cases where it would still be connected to a coworker laptop when exchanging devices in the office.
I’m keeping note that the process is not very user-friendly and will bring that up in our internal dev discussion.
As for the Unity apps, I believe @oliver would have better answers than me.
Ok, so I have connected the pen to another laptop before and I’m guessing this might be why I’m having trouble finding it in the dashboard app.
How do I restart the pen?
Hi William,
The Handle prototype can only connect to one system at a time. To connect to another laptop, you must remove the handle from the list of Bluetooth devices on the previous laptop.
Attached is the Tracked Handle Start Guide - for more detail on the Handle let me know if this helps!
Tracked Handle - Start Guide.pdf (351.8 KB)
Hello guys!
My pen was connected to another notebook that was being used far away and I finally got a chance to unpair the pen from the notebook it was connected for the first time, but I still wasn’t able to connect it to my desktop despite being able to connect it to the notebook.
Bluetooth seems to be working fine in my desktop.
Is there any way to reset the pen hardware or is there any specific way to unpair the pen (I was trying removing it from de devices list in the windows devices settings).
Hello William,
My first guess would be that the bluetooth adapter may be the source of the problem.
To help us debug, could you open the registry editor (Windows button> type regedit) and navigate to the following folder: HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\SERIALCOMM. Once there, take a screenshot of the keys with your handle paired and with your handle not paired (so we can tell the difference). As an example:
Additionally, if you could access the device manager (windows buttons > type device manager) and open the Ports (COM & LPT) node and again, take a screenshot with your handle paired and with your handle not paired. As an example:
For context, this is the registry location that is currently being checked by our APIs to automatically detect handles and inverse3 devices and a missing entry could explain why we’re not automatically detecting the handle.
Thanks,
Remi A.
Here is the image with the pen paired:
And this is the image with it unpaired:
It seems COM14 port is the inverse3 robot, as seen from the dashboard app.
Hello William,
Thanks for the screenshot. From what I can see, Windows is correctly detecting the pen and providing the COM ports where we’re looking for them so I don’t think the adapter is the issue. After having re-checked a bunch of code in the hardware APIs, I think the issue you’re running into is one that I’ve recently manage to isolate and reproduce.
The short version is that the bluetooth drivers within the handle tends to get itself into a bad state when the hardware API doesn’t correctly close the connection with the handle (e.g. when an application crashes). In this state, the handle is essentially unusable and the only recovery mechanism that seems to work is to do a power cycle (i.e. let the batteries run out). Unfortunately, the bluetooth drivers is not something we have control over which limits our ability to fix this specific problem.
In the long term, we’re looking at alternatives to bluetooth as our wireless protocol that will better suited to our needs. In the short term, we’re currently testing a wired firmware for the handle which, so far, has been far more reliable then bluetooth but will require that the handle be plugged in to be usable. It might also require some light changes in your program but these details haven’t been nailed down just yet.
Unfortunately, I don’t think the new firmware and the hardware API is ready to be released just yet so, in the meantime, my only recommendation is to let the batteries on your handle run out and then try again connecting to it again. I’ll post an update in this thread when we’re closer to being able to release the short term solution.
Cheers,
Remi A.
Thanks, it seems making a power cycle worked as a reset for the pen.
So, now the pen seems to be working inside the dashboard unity app, however, the update rate is dropping below 40Hz and sometimes it freezes the orientation for some seconds.
I still couldn’t figure out how to use the unity package API to connect the handle. Could you guys send me any project (it can even be the dashboard) uncompiled as an example of the pen usage?
Hey William, looks to me like it may be a battery issue. The behaviour you describe sounds like the behaviour I’ve observed when my wireless handle is low on juice, or has a bad battery connection. Do you observe the same thing if it’s plugged in to power?
The rate drops to 40Hz were observed when the pen was plugged in the charger. When it was unplugged, the complete freezes on the orientation were more frequent and longer.
Hey William, the guidance I’m getting from our team is that users with our wireless prototype should wait for our wired prototype which will ship alongside an API update and will also eventually support higher update rates (and will obviously not suffer from the power issues).
Hi there!
I still really need the pen orientation queries for the project I’m working on, could you please provide the sample project for the pen to test how it works inside the project even with the pen update rate drops.
I’m making a version of the projects that sets the orientation manually, but it would be very nice to have the project completely integrated with all the robot features.
Hello William,
There’s unfortunately no direct support for handles in the Unity plugin at the moment. The situation is being worked on and should hopefully be remedied in the near future.
In the interim, you can use the C# API directly to communicate with the handle. While I’m not very familiar with C# and Unity, I was able to extract the following code snippet that should allow you to access the orientation data from the handle:
using Haply.IO.Ports;
using Haply.HardwareAPI.Runtime.Generated;
using ( var serialPort = new SerialPort( "COMX" ) )
{
serialPort.Open();
serialPort.DiscardInBuffer();
serialPort.DiscardOutBuffer();
using ( var connection = new HandleMessageBus( serialPort.BaseStream, 1024 ) )
{
void HandleReceiveToolStatusMessage ( HandleStatusMessage m )
{
m.orientationX;
m.orientationY;
m.orientationZ;
m.orientationW;
m.hallEffectSensorLevel;
m.userData;
}
connection.onReceiveHandleStatusMessage += HandleReceiveToolStatusMessage;
while ( true ) {}
}
}
This isn’t much but hopefully it should unblock you until we have a more permanent solution that you can work with.
Thanks, it worked inside one of the projects, but it’s giving an “IOException Access denied” on the SerialPort.Open() function in a project that was made in an older Unity version.
Also, is there any code snippet to calibrate the pen orientation inside unity?
In my limited experience of using the WIn32 API to manage serial ports, the access denied error occurs if there’s another open handle on the serial port. Windows only allows a single open handle for any given port so to solve this issue you’ll need to figure out what else on your computer has an open handle on the port and kill that process. If you’re having a hard time finding the rogue process, you can use Process Explorer - Sysinternals | Microsoft Learn which has a tool to do system wide search for handles.
As for handle calibration, we have a public C++ example that contains some calibration logic:
Where MyHandle::Calibrate should be called when the user has positioned the handle flat on a table facing the screen and MyHandle::GetStatusResponse is where the latest handle’s quaternion is offset with the calibration quaternion. I’m also working on a new example for the C++ API that contains a bit more explanation and has a calibration workflow built in:
Hope that helps.