[Download] libhpcalcs: a toolkit for communicating with Prime calcs...
#1

Over the past month or so, I've been working on reverse-engineering Prime communications (thanks to packet dumps sent by critor), and from that knowledge, implementing a backend for a FLOSS third-party Prime connectivity kit aimed at Windows, Linux and MacOS X (possibly FreeBSD and derivatives as well, the USB HID library I've chosen to use works on FreeBSD).



I've posted about it in the traditionally TI community (TI-Planet, Omnimaga, Cemetech), but not in the HP community, as I didn't feel it's fully baked yet... but now that Edwin Ried has announced he's working on the same, I'd like to reduce reverse-engineering duplication work for him, and also, propose a more portable code base :)



libhpcalcs currently has support for the following operations, through a terminal-based interface ("test_hpcalcs"):

  • sending files;
  • receiving files;
  • receiving backups;
  • receiving screenshots;
  • getting undecoded information about the calculator, including firmware version;
  • setting the calculator's time;
  • triggering some form of "ready" check, or something like that;

Development source code: https://github.com/debrouxl/hplp (master branch is stable, master2 contains changes for testing, which may be rewritten before integration to master)

Ready-made install script aimed at *nix: https://raw.github.com/debrouxl/hplp/master/install_hplp.sh. Unless your distro packages hidapi (Debian and derivatives do not), you'll have to compile it yourself (autotools-based program, so configure && make && sudo make install).

Latest Windows binaries + source tarball: http://tiplanet.org/beta/libhpcalcs-0.0.1-package.zip .

EDIT: Program to be run after decompressing or compiling: test_hpcalcs(.exe).





The current state of the code base (written in C99 for interoperability and portability, BTW) is that:

  • on the plus side, the aforementioned operations usually work, for my several beta-testers, on Windows, Linux and MacOS X. One of the operations is reported to choke on MacOS X, but it works on Linux and Windows;
  • on the minus side, there's no post-processing of screenshots so that they have proper colors... and most of all, there's no GUI...
From the POV of most end users, lack of a GUI makes the program worthless. That's why libhpcalcs wasn't announced outside of the traditionally TI community yet. I don't know how people here would react to working, but arguably hard to use for a significant portion of the target user base, code... therefore, as a newcomer, better staying quiet than spamming people with my programs too early, or so I thought :)



The architecture and API are derived from the time-proven libti* architecture. They're likely to be able to support the 39gII as well (should someone provide descriptor and packet dumps), which is closer to the Prime than, say, a TI-85 with BlackLink cable is to a Nspire with USB cable.



The Prime might support more operations, but that's undetermined yet. For instance (those are operations available on some, or all, TI graphing calculators):

  • quickly getting the list of files without triggering a lengthy full backup ("dirlist", an very important operation - IMO, if HP hasn't implemented it yet, they should do it !);
  • sending keypresses;
  • remote file operations (delete, rename, maybe even calculator-side copy).


Tim Wessman needs to be thanked once more for publicly helping with the peculiarities of CRC computations and screenshots on the Prime.

Edited: 18 Nov 2013, 1:20 a.m. after one or more responses were posted

#2

Cool! Great work, I started to learn about how to sniff the USB last week but I got stuck when tried to find the CRC. Attempted via reverse engineering the exe but since it is kinda a wrapper for something (QT?) it is really a mess to follow. Then I noticed the CRC was not changing for all the data... and then, few days ago, that the calculator just accepts any CRC so I hardcoded it hahaha!

You have added so much extra functionality I think I may build the .net wrapper/pinvoke and gui so the windows utilities starts to pop like the plague.

BTW I will write a small article in spanish for my website http://ried.cl how it is the process of doing this kind of task, mainly because it is something I usually see happening: you are stuck sometimes with the manufacturers utilities (like what happens on laboratory equipment... I was stuck with my oscilloscope crappy app: http://www.eevblog.com/forum/testgear/fixing-easyscopex/ and multitester crappy app :D)

#3

Nice work. Your DOS-box app seems to run fine, until I try to send a program to the HP Prime. I just get a leading control character (a small square) and nothing else. What am I doing wrong, or it is a bug?

Namir

#4

Hi Erwin,



> Cool! Great work,

You're welcome. I've spent most of my free time on it since nearly a month ago (and consumed some free time of my beta-testers, I don't even have a Prime myself) :)



> I started to learn about how to sniff the USB last week but I got stuck when tried to find the CRC.

Yeah, same for me, until Tim Wessman helped. I even wrote and published an OpenMP-based brute-forcing program, but it never produced the formula, so I assume there was a bug in it.

BTW, what sniffer did you use ? Wireshark started understanding files produced by usbpcap several months ago, so it's easier to make sense of USB dumps than it used to be.



> Attempted via reverse engineering the exe but since it is kinda a wrapper for something (QT?) it is really a mess to follow.

Yeah, plus even if it falls under the "reverse-engineering for interoperability purposes" exception as well, reverse-engineering executable binaries is slightly less safe from a legal POV than so-called "clean-room reverse-engineering" by examining what goes over the wire.

> Then I noticed the CRC was not changing for all the data...

Yup.

> and then, few days ago, that the calculator just accepts any CRC so I hardcoded it hahaha!

Didn't notice that :)



> You have added so much extra functionality I think I may build the .net wrapper/pinvoke and gui so the windows utilities starts to pop like the plague.

I agree that building a wrapper (which other .net-based tools could leverage) would be quicker than re-coding the functionality contained in libhpcalcs + adding the same amount of code comments.

One of the reasons why I used a pure C API is that anything can interoperate with C. All of the main clients of libti* (TILP, TIEmu, titools, TilEm) are written in C. However, for libhpcalcs, given that nowadays, the main choice for a portable UI + OS abstraction layer toolkit is Qt, there shall certainly be a C++ client of libhpcalcs.



> BTW I will write a small article in spanish for my website http://ried.cl how it is the process of doing this kind of task [...]

Sounds good.





On my side, locally, I've just added a dependency on libpng 1.6 to libhpcalcs, and I'm going to read again the simplified libpng 1.6 API, a boon for easier libpng usage.





Bye, Lionel.

#5

I assume you get this character after entering the file name ?

In general, on Windows, the program produces a "trace.txt" file for detailed debugging purposes (*). The trace.txt file may be too large for posting here, but you could always send it to me by e-mail :)



*: redirecting trace output to a file on Windows is a consequence of the fantastic slowness of the Windows console. libti*/tilp do the same. When producing the same output in the Windows console, due to the timing sensitivity of USB HID transfers (interrupt mode), the program literally misses 3 packets out of 4 on a ~20 KB screenshot (!), while it is very stable when outputting to a file (otherwise I wouldn't have dared publishing it :D).

#6

Some HP guys seem to read this forum sometimes, they might ask HP to send you a xmas gift :) (TW?, the spreadsheet french guy? :D)

I used USBTrace. UI is crappy but sniffing stuff is top notch... and you **might** be able to remove the trial limitation with ollydbg in 5 mins.

I still haven't decoded all the format types (don't think I need too). Will you publish any documentation of the structures you found? I just wanted to be able to send programs easily without the clunkiness of the HP Conn Kit. But what I still need to add is the send via command line arguments (double click over a hpprgm) and I am done for my needs.

Edited: 17 Nov 2013, 10:40 a.m.

#7

> Some HP guys seem to read this forum sometimes, they might ask HP to send you a xmas gift :)

Yeah, at least two members of the Prime development team, and several HP beta-testers, attend this forum and interact with users.

If HP were to send me a Prime calculator, it would be my 25th graphing calculator. All current 24 calculators are TI models, used more or less frequently for working on libti*.



> I used USBTrace. UI is crappy but sniffing stuff is top notch...

OK. usbpcap has good sniffing, and Wireshark has good UI.

> and you **might** be able to remove the trial limitation with ollydbg in 5 mins.

No need :)



> I still haven't decoded all the format types (don't think I need too).

Indeed, I have decoded most of them from USB dumps of backup. I'm waiting for feedback on .hpappnote and .hpappprgm ( http://tiplanet.org/forum/viewtopic.php?p=152416#p152416 and below).



> Will you publish any documentation of the structures you found?

Sure, but after implementing PNG color conversion and improving the way backup works (*) :)

The main header files already contain a wealth of comments in Doxygen format.



> I just wanted to be able to send programs easily without the clunkiness of the HP Conn Kit. But what I still need to add is the send via command line arguments (double click over a hpprgm) and I am done for my needs.

Yup, there's a clear user need for simple GUI programs, simple CLI programs, and programs which work on Linux and MacOS X.





*: for recv_backup, instead of receiving files one by one, the library should attempt to read as much data as possible, then split data. Without the occasional packet loss in the USB HID USB stack (occurs once in a while on Windows despite the delays added by HP), there's no difference, but when one or multiple packets were lost, the new way will make it possible to retrieve more files.

#8

Hi debrouxl,
this is great news. Being a Linux-only user, my usage of the Prime has been seriously crippled, since the HP connect tool doesn't work under Wine, and on-calc program writing is inconvenient. I really appreciate your efforts.
I'm trying to follow the instructions to download and compile your program.
Thanks again! :)

Cristian

#9

Quote:
If HP were to send me a Prime calculator, it would be my 25th graphing calculator. All current 24 calculators are TI models, used more or less frequently for working on libti*.




You need to discard all that junk and get a 50g, a Prime and probably some old 48 for nostalgy.

#10

OK, downloaded and installed with dependencies on the latest Ubuntu. Now what - man page? Help file? What to run and with what syntax/switches?

#11

The README indicates that "tests/test_hpcalcs" should be run :)

I agree that I should have mentioned this above, like I did on TI-Planet, Omnimaga and Cemetech... I'm going to edit the post above.

#12

Geir... I've tried installing the program, but after some dependency problems I gave up for lack of time.
I too am running the latest Ubuntu... Would it be possible for you to post a step-by-step description of what you did to install it?

Cristian

#13

If the missing dependency is hidapi, you need to download and install it yourself (autoreconf -i -f && configure && make && sudo make install), as indicated above and in the README :)

I used hidapi because it works with the same API on Windows, Linux, MacOS X and FreeBSD. As far as I could see, few HID libraries have such portability...



Other dependencies are packaged by Debian / Ubuntu, the package names are printed on the screen by the install_hplp.sh script at the very beginning of its execution.

Edited: 18 Nov 2013, 3:27 p.m.

#14

I'm not going to get rid of my set of TI calculators as long as I'm the maintainer of libti*, i.e. the main, and only maintained, backend for third-party "linking software" aimed at TI-Z80, TI-68k and TI-Nspire calculators ;)



Possibly Related Threads…
Thread Author Replies Views Last Post
  HP Prime Emulator download John Colvin 2 2,047 12-14-2013, 05:54 PM
Last Post: John Colvin
  [Download] PrimeComm: Alternative small utility to upload/download programs to the HP Prime Erwin Ried 6 2,475 11-17-2013, 10:18 AM
Last Post: Erwin Ried
  Swiss Micros calcs Mike Powell 6 2,329 10-26-2013, 05:03 PM
Last Post: Mike Powell
  Help me alert regular users to dangers to their calcs Bruce Larrabee 12 3,392 10-06-2013, 08:30 PM
Last Post: Bruce Larrabee
  USB Chargers for HP calcs Matt Agajanian 3 1,523 08-18-2013, 10:58 PM
Last Post: Craig Ruff
  Look on Fournier pressure sensor for HP calcs Mic 0 925 04-11-2013, 12:57 PM
Last Post: Mic
  Safe charging LED HP calcs Anoop Sahal 6 2,235 03-29-2013, 09:08 AM
Last Post: Anoop Sahal
  HP Calcs & 'NUMB3RS' Matt Agajanian 1 1,092 11-11-2012, 10:07 AM
Last Post: Eddie W. Shore
  Official revival of old calcs wildpig 7 2,360 08-20-2012, 04:11 PM
Last Post: Ingo
  DM calcs arrived, but... David Ramsey 4 1,823 08-04-2012, 12:42 PM
Last Post: Michael de Estrada

Forum Jump: