Smalltalk › Squeak › Squeak - Dev

[ANN] prototype for reading National Instrument TDMS files

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

3 messages Options

Nicolas Cellier

[ANN] prototype for reading National Instrument TDMS files

Hi all,

I have published NITDMSReader, a prototype and proof of concept for reading National Instrument TDMS data files.

Those binary files are produced by N.I. softwares like Labview and are mainly used for data acquisition of various sensors.

The interesting bytes is that I tried to carry bulk data transfer via BitBlt.

I published a BulkDataTransfer package for that too. I think it may be of general use for transfer of bytes across bit arrays.

An example usage is:

AndreasSystemProfiler spyOn:

    [NITDMSFileReader new
         filename: 'c:\MyPath\MyFile.tdms';
         scan;
         readValuesForChannel: #('MyGroup' 'MyChannel').]

I can extract one channel of 60k records from a 16MBytes file in approximately half a second, but Time seems dominated by file access.

This is all on STEM, http://www.squeaksource.com/STEM.html.

For now the package rely on BinaryStream, MultipleDimensionArray and on my inbox Collection refactorings (Long64Array) and recent Float64Array too, as well as ExternalFloatComplexArray from Smallapack... If ever you try to lad the package, don't be surprised to see a few undeclared. I'll clean that later and will provide some Metacello recipe.

marcel.taeumel

Re: [ANN] prototype for reading National Instrument TDMS files

Hi Nicolas! :-)

Thanks, this looks nice. I wasn't aware of the STEM repo until now. We should really think about a Geometry/Math package for Squeak Trunk. ^___^ I should write down some thoughts on this matter...

Best,

Marcel

Am 04.05.2020 05:30:48 schrieb Nicolas Cellier <[hidden email]>:

Hi all,
I have published NITDMSReader, a prototype and proof of concept for reading National Instrument TDMS data files.
Those binary files are produced by N.I. softwares like Labview and are mainly used for data acquisition of various sensors.

The interesting bytes is that I tried to carry bulk data transfer via BitBlt.
I published a BulkDataTransfer package for that too. I think it may be of general use for transfer of bytes across bit arrays.

An example usage is:
AndreasSystemProfiler spyOn:
    [NITDMSFileReader new
         filename: 'c:\MyPath\MyFile.tdms';
         scan;
         readValuesForChannel: #('MyGroup' 'MyChannel').]
I can extract one channel of 60k records from a 16MBytes file in approximately half a second, but Time seems dominated by file access.

This is all on STEM, http://www.squeaksource.com/STEM.html.
For now the package rely on BinaryStream, MultipleDimensionArray and on my inbox Collection refactorings (Long64Array) and recent Float64Array too, as well as ExternalFloatComplexArray from Smallapack... If ever you try to lad the package, don't be surprised to see a few undeclared. I'll clean that later and will provide some Metacello recipe.

David T. Lewis

Re: [ANN] prototype for reading National Instrument TDMS files

In reply to this post by Nicolas Cellier

Very cool! Thank you.

Dave

On Mon, May 04, 2020 at 05:29:30AM +0200, Nicolas Cellier wrote:

> Hi all,
> I have published NITDMSReader, a prototype and proof of concept for reading
> National Instrument TDMS data files.
> Those binary files are produced by N.I. softwares like Labview and are
> mainly used for data acquisition of various sensors.
>
> The interesting bytes is that I tried to carry bulk data transfer via
> BitBlt.
> I published a BulkDataTransfer package for that too. I think it may be of
> general use for transfer of bytes across bit arrays.
>
> An example usage is:
> AndreasSystemProfiler spyOn:
> [NITDMSFileReader new
> filename: 'c:\MyPath\MyFile.tdms';
> scan;
> readValuesForChannel: #('MyGroup' 'MyChannel').]
> I can extract one channel of 60k records from a 16MBytes file in
> approximately half a second, but Time seems dominated by file access.
>
> This is all on STEM, http://www.squeaksource.com/STEM.html.
> For now the package rely on BinaryStream, MultipleDimensionArray and on my
> inbox Collection refactorings (Long64Array) and recent Float64Array too, as
> well as ExternalFloatComplexArray from Smallapack... If ever you try to lad
> the package, don't be surprised to see a few undeclared. I'll clean that
> later and will provide some Metacello recipe.