File size limit

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

File size limit

Chantal Thibodeau
Hi
 
We are using VSE 3.0 for over 15 years and in this period, files are getting bigger and bigger.  We are unable to read files larger than 4 gigs.  At writing, there is no problem.
 
I know this is because of the "maxInteger" value.
 
Is there a way to bypass that limit (except splitting larger files in two) ?
 
Thanks
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Andreas Rosenberg
Hi!
 
The Win32 API does support 64 bit file pointers.
Currently the methods calling the API use zero for the upper 32 bit. This limits the files to 4GB.
I think it should be possible to enhance the File related classes to support the 64 bit file pointers.

See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365541(v=vs.85).aspx
 
Perhaps somebody already did this and is willing to share it?
 
Regards,
   Andreas
 

Andreas Rosenberg | eMail: [hidden email]
APIS GmbH         | Phone: +49 9482 9415-0
Im Haslet 42      | Fax: +49 9482 9415-55
93086 Wörth/D     | WWW:
<http://www.apis.de/>
Germany           | <http://www.fmea.de/>

 
-----Original Message-----
From: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]]On Behalf Of Chantal Thibodeau
Sent: Montag, 13. Februar 2012 14:55
To: [hidden email]
Subject: File size limit

Hi
 
We are using VSE 3.0 for over 15 years and in this period, files are getting bigger and bigger.  We are unable to read files larger than 4 gigs.  At writing, there is no problem.
 
I know this is because of the "maxInteger" value.
 
Is there a way to bypass that limit (except splitting larger files in two) ?
 
Thanks
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Leandro Caniglia
Try

FileHandle >>position: anInteger

| large return |

large := WinLargeInteger fromInteger: anInteger.

return := KernelLibrary

setFilePointer: self asParameter

offset: (large bytesAtOffset: 0 count: 4)

offsetHigh: (large bytesAtOffset: 4 count: 4)

move: FileBegin.

return = 16rFFFFFFFF ifTrue: [^self osError]




On Mon, Feb 13, 2012 at 12:07 PM, Andreas Rosenberg <[hidden email]> wrote:
Hi!
 
The Win32 API does support 64 bit file pointers.
Currently the methods calling the API use zero for the upper 32 bit. This limits the files to 4GB.
I think it should be possible to enhance the File related classes to support the 64 bit file pointers.

See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365541(v=vs.85).aspx
 
Perhaps somebody already did this and is willing to share it?
 
Regards,
   Andreas
 

Andreas Rosenberg | eMail: [hidden email]
APIS GmbH         | Phone: +49 9482 9415-0
Im Haslet 42      | Fax: +49 9482 9415-55
93086 Wörth/D     | WWW:
<http://www.apis.de/>
Germany           | <http://www.fmea.de/>

 
-----Original Message-----
From: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]]On Behalf Of Chantal Thibodeau
Sent: Montag, 13. Februar 2012 14:55
To: [hidden email]
Subject: File size limit

Hi
 
We are using VSE 3.0 for over 15 years and in this period, files are getting bigger and bigger.  We are unable to read files larger than 4 gigs.  At writing, there is no problem.
 
I know this is because of the "maxInteger" value.
 
Is there a way to bypass that limit (except splitting larger files in two) ?
 
Thanks
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Chantal Thibodeau
In reply to this post by Andreas Rosenberg
Thanks, i'll wait for an answer
----- Original Message -----
Sent: Monday, February 13, 2012 10:07 AM
Subject: Re: File size limit

Hi!
 
The Win32 API does support 64 bit file pointers.
Currently the methods calling the API use zero for the upper 32 bit. This limits the files to 4GB.
I think it should be possible to enhance the File related classes to support the 64 bit file pointers.

See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365541(v=vs.85).aspx
 
Perhaps somebody already did this and is willing to share it?
 
Regards,
   Andreas
 

Andreas Rosenberg | eMail: [hidden email]
APIS GmbH         | Phone: +49 9482 9415-0
Im Haslet 42      | Fax: +49 9482 9415-55
93086 Wörth/D     | WWW:
<http://www.apis.de/>
Germany           | <http://www.fmea.de/>

 
-----Original Message-----
From: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]]On Behalf Of Chantal Thibodeau
Sent: Montag, 13. Februar 2012 14:55
To: [hidden email]
Subject: File size limit

Hi
 
We are using VSE 3.0 for over 15 years and in this period, files are getting bigger and bigger.  We are unable to read files larger than 4 gigs.  At writing, there is no problem.
 
I know this is because of the "maxInteger" value.
 
Is there a way to bypass that limit (except splitting larger files in two) ?
 
Thanks
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Leandro Caniglia
Be careful. I haven't tried the hint myself yet.

On Mon, Feb 13, 2012 at 1:06 PM, Chantal Thibodeau <[hidden email]> wrote:
Thanks, i'll wait for an answer
----- Original Message -----
Sent: Monday, February 13, 2012 10:07 AM
Subject: Re: File size limit

Hi!
 
The Win32 API does support 64 bit file pointers.
Currently the methods calling the API use zero for the upper 32 bit. This limits the files to 4GB.
I think it should be possible to enhance the File related classes to support the 64 bit file pointers.

See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365541(v=vs.85).aspx
 
Perhaps somebody already did this and is willing to share it?
 
Regards,
   Andreas
 

Andreas Rosenberg | eMail: [hidden email]
APIS GmbH         | Phone: +49 9482 9415-0
Im Haslet 42      | Fax: +49 9482 9415-55
93086 Wörth/D     | WWW:
<http://www.apis.de/>
Germany           | <http://www.fmea.de/>

 
-----Original Message-----
From: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]]On Behalf Of Chantal Thibodeau
Sent: Montag, 13. Februar 2012 14:55
To: [hidden email]
Subject: File size limit

Hi
 
We are using VSE 3.0 for over 15 years and in this period, files are getting bigger and bigger.  We are unable to read files larger than 4 gigs.  At writing, there is no problem.
 
I know this is because of the "maxInteger" value.
 
Is there a way to bypass that limit (except splitting larger files in two) ?
 
Thanks
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management *** *** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Leandro Caniglia
More info.

There are two additional methods you should also change:

FileHandle >> #endByte

| large success |

large := WinLargeInteger new.

success := KernelLibrary getFileSize: self asParameter fileSizeHigh: large.

success ifFalse: [self osError].

^large asUnsignedInteger


KernelDLL >> #getFileSize: aFileHandle fileSizeHigh: aWinLargeInteger

<api: GetFileSizeEx ulong struct boolean>

^self invalidArgument


On Mon, Feb 13, 2012 at 1:21 PM, Leandro Caniglia <[hidden email]> wrote:
Be careful. I haven't tried the hint myself yet.


On Mon, Feb 13, 2012 at 1:06 PM, Chantal Thibodeau <[hidden email]> wrote:
Thanks, i'll wait for an answer
----- Original Message -----
Sent: Monday, February 13, 2012 10:07 AM
Subject: Re: File size limit

Hi!
 
The Win32 API does support 64 bit file pointers.
Currently the methods calling the API use zero for the upper 32 bit. This limits the files to 4GB.
I think it should be possible to enhance the File related classes to support the 64 bit file pointers.

See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365541(v=vs.85).aspx
 
Perhaps somebody already did this and is willing to share it?
 
Regards,
   Andreas
 

Andreas Rosenberg | eMail: [hidden email]
APIS GmbH         | Phone: +49 9482 9415-0
Im Haslet 42      | Fax: +49 9482 9415-55
93086 Wörth/D     | WWW:
<http://www.apis.de/>
Germany           | <http://www.fmea.de/>

 
-----Original Message-----
From: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]]On Behalf Of Chantal Thibodeau
Sent: Montag, 13. Februar 2012 14:55
To: [hidden email]
Subject: File size limit

Hi
 
We are using VSE 3.0 for over 15 years and in this period, files are getting bigger and bigger.  We are unable to read files larger than 4 gigs.  At writing, there is no problem.
 
I know this is because of the "maxInteger" value.
 
Is there a way to bypass that limit (except splitting larger files in two) ?
 
Thanks
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management *** *** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***


*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Chantal Thibodeau
Thanks, I'm looking into that (the first modification wasn't enough).
 
I'm using a <nextLine> ou <upTo:> to parse the file.
 
I will get back to you later this afternoon.
 
And thanks again
----- Original Message -----
Sent: Monday, February 13, 2012 12:15 PM
Subject: Re: File size limit

More info.

There are two additional methods you should also change:

FileHandle >> #endByte

| large success |

large := WinLargeInteger new.

success := KernelLibrary getFileSize: self asParameter fileSizeHigh: large.

success ifFalse: [self osError].

^large asUnsignedInteger


KernelDLL >> #getFileSize: aFileHandle fileSizeHigh: aWinLargeInteger

<api: GetFileSizeEx ulong struct boolean>

^self invalidArgument


On Mon, Feb 13, 2012 at 1:21 PM, Leandro Caniglia <[hidden email]> wrote:
Be careful. I haven't tried the hint myself yet.


On Mon, Feb 13, 2012 at 1:06 PM, Chantal Thibodeau <[hidden email]> wrote:
Thanks, i'll wait for an answer
----- Original Message -----
Sent: Monday, February 13, 2012 10:07 AM
Subject: Re: File size limit

Hi!
 
The Win32 API does support 64 bit file pointers.
Currently the methods calling the API use zero for the upper 32 bit. This limits the files to 4GB.
I think it should be possible to enhance the File related classes to support the 64 bit file pointers.

See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365541(v=vs.85).aspx
 
Perhaps somebody already did this and is willing to share it?
 
Regards,
   Andreas
 

Andreas Rosenberg | eMail: [hidden email]
APIS GmbH         | Phone: +49 9482 9415-0
Im Haslet 42      | Fax: +49 9482 9415-55
93086 Wörth/D     | WWW:
<http://www.apis.de/>
Germany           | <http://www.fmea.de/>

 
-----Original Message-----
From: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]]On Behalf Of Chantal Thibodeau
Sent: Montag, 13. Februar 2012 14:55
To: [hidden email]
Subject: File size limit

Hi
 
We are using VSE 3.0 for over 15 years and in this period, files are getting bigger and bigger.  We are unable to read files larger than 4 gigs.  At writing, there is no problem.
 
I know this is because of the "maxInteger" value.
 
Is there a way to bypass that limit (except splitting larger files in two) ?
 
Thanks
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management *** *** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***


*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

AW: File size limit

Möbus. Manfred

Hi,

In addition to the methods already mentioned

FileHandle>>endByte,

FileHandle>>position:,

 

we have also changed the methods

FileHandle>>position,

FileHandle>>readInto:atPage:pageSize

FileHandle>>writeFrom:toPage:for:pageSize

 

using the same type of enhancements as given by Leandro.

 

position

       "(Private) Private - Answer the current file position."

 

       | pos high error |

       high := ExternalLong fromInteger: 0.

       ( pos := KernelLibrary

             setFilePointer: self asParameter

             offset: 0

             offsetHigh: high asParameter

             move: FileCurrent ) = 16rFFFFFFFF

                    ifTrue: [ #sdgHack. "^self osError"

                           (error := self lastOsError) ~= 0 ifTrue: [

                                  ^self osError: error in: #SetFilePointer ] ].

       high asInteger = 0 ifTrue: [  "keine high-bytes"

             ^pos ].

       ^(high asInteger bitShift: 32) + pos

 

readInto: aString atPage: anInteger pageSize: sizeInteger

       "(Private) Private - Read a page (sizeInteger bytes) or less

             (if at end of file) at page number anInteger from

             the receiver file into aString.  Answer the number

             of bytes read.     aString size must be >= sizeInteger."

 

       | long |

       self position: anInteger * sizeInteger.

       long := ExternalLong fromInteger: 0.

       ( KernelLibrary

             readFile: self asParameter

             buffer: aString

             bytesToRead: sizeInteger

             bytesRead: long asParameter

             overlapped: nil )

                    ifFalse: [ #sdgHack. "^self osError"

                           ^self osErrorIn: #ReadFile ].

       ^long asInteger

 

writeFrom: aString toPage: anInteger for: size pageSize: sizeInteger

       "(Private) Private - Write size bytes of aString to

             the receiver file at page number anInteger.

             Page size is sizeInteger."

 

       | long |

       self position: anInteger * sizeInteger.

       long := ExternalLong fromInteger: 0.

       ( KernelLibrary

             writeFile: self asParameter

             buffer: aString

             bytesToWrite: size

             bytesWritten: long asParameter

             overlapped: nil )

                    ifFalse: [ #sdgHack. "^self osError"

                           ^self osErrorIn: #WriteFile ]

 

Manfred.

 

Von: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]] Im Auftrag von Chantal Thibodeau
Gesendet: Montag, 13. Februar 2012 19:10
An: [hidden email]
Betreff: Re: File size limit

 

Thanks, I'm looking into that (the first modification wasn't enough).

 

I'm using a <nextLine> ou <upTo:> to parse the file.

 

I will get back to you later this afternoon.

 

And thanks again

----- Original Message -----

Sent: Monday, February 13, 2012 12:15 PM

Subject: Re: File size limit

 

More info.

 

There are two additional methods you should also change:

 

FileHandle >> #endByte

| large success |

large := WinLargeInteger new.

success := KernelLibrary getFileSize: self asParameter fileSizeHigh: large.

success ifFalse: [self osError].

^large asUnsignedInteger

 

KernelDLL >> #getFileSize: aFileHandle fileSizeHigh: aWinLargeInteger

<api: GetFileSizeEx ulong struct boolean>

^self invalidArgument

 

On Mon, Feb 13, 2012 at 1:21 PM, Leandro Caniglia <[hidden email]> wrote:

Be careful. I haven't tried the hint myself yet.

 

On Mon, Feb 13, 2012 at 1:06 PM, Chantal Thibodeau <[hidden email]> wrote:

Thanks, i'll wait for an answer

----- Original Message -----

Sent: Monday, February 13, 2012 10:07 AM

Subject: Re: File size limit

 

Hi!

 

The Win32 API does support 64 bit file pointers.

Currently the methods calling the API use zero for the upper 32 bit. This limits the files to 4GB.

I think it should be possible to enhance the File related classes to support the 64 bit file pointers.

See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365541(v=vs.85).aspx

 

Perhaps somebody already did this and is willing to share it?

 

Regards,

   Andreas

 

Andreas Rosenberg | eMail: [hidden email]
APIS GmbH         | Phone: +49 9482 9415-0
Im Haslet 42      | Fax: +49 9482 9415-55
93086 Wörth/D     | WWW: <http://www.apis.de/>

Germany           | <http://www.fmea.de/>

 

-----Original Message-----
From: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]]On Behalf Of Chantal Thibodeau
Sent: Montag, 13. Februar 2012 14:55
To: [hidden email]
Subject: File size limit

Hi

 

We are using VSE 3.0 for over 15 years and in this period, files are getting bigger and bigger.  We are unable to read files larger than 4 gigs.  At writing, there is no problem.

 

I know this is because of the "maxInteger" value.

 

Is there a way to bypass that limit (except splitting larger files in two) ?

 

Thanks

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management *** *** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

 

 

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***



Treffen Sie AEB vom 13.-15. März 2012 auf der LogiMAT in Stuttgart. Halle 7, Stand 261.
Vereinbaren Sie jetzt einen Termin und erhalten eine Eintrittskarte: www.aeb.de/logimat
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Leandro Caniglia
Thanks Manfred!

I liked your catch of FileHandle>>#position and have a question: Is this method ever used?

/Leandro

On Mon, Feb 13, 2012 at 6:17 PM, Möbus. Manfred <[hidden email]> wrote:

Hi,

In addition to the methods already mentioned

FileHandle>>endByte,

FileHandle>>position:,

 

we have also changed the methods

FileHandle>>position,

FileHandle>>readInto:atPage:pageSize

FileHandle>>writeFrom:toPage:for:pageSize

 

using the same type of enhancements as given by Leandro.

 

position

       "(Private) Private - Answer the current file position."

 

       | pos high error |

       high := ExternalLong fromInteger: 0.

       ( pos := KernelLibrary

             setFilePointer: self asParameter

             offset: 0

             offsetHigh: high asParameter

             move: FileCurrent ) = 16rFFFFFFFF

                    ifTrue: [ #sdgHack. "^self osError"

                           (error := self lastOsError) ~= 0 ifTrue: [

                                  ^self osError: error in: #SetFilePointer ] ].

       high asInteger = 0 ifTrue: [  "keine high-bytes"

             ^pos ].

       ^(high asInteger bitShift: 32) + pos

 

readInto: aString atPage: anInteger pageSize: sizeInteger

       "(Private) Private - Read a page (sizeInteger bytes) or less

             (if at end of file) at page number anInteger from

             the receiver file into aString.  Answer the number

             of bytes read.     aString size must be >= sizeInteger."

 

       | long |

       self position: anInteger * sizeInteger.

       long := ExternalLong fromInteger: 0.

       ( KernelLibrary

             readFile: self asParameter

             buffer: aString

             bytesToRead: sizeInteger

             bytesRead: long asParameter

             overlapped: nil )

                    ifFalse: [ #sdgHack. "^self osError"

                           ^self osErrorIn: #ReadFile ].

       ^long asInteger

 

writeFrom: aString toPage: anInteger for: size pageSize: sizeInteger

       "(Private) Private - Write size bytes of aString to

             the receiver file at page number anInteger.

             Page size is sizeInteger."

 

       | long |

       self position: anInteger * sizeInteger.

       long := ExternalLong fromInteger: 0.

       ( KernelLibrary

             writeFile: self asParameter

             buffer: aString

             bytesToWrite: size

             bytesWritten: long asParameter

             overlapped: nil )

                    ifFalse: [ #sdgHack. "^self osError"

                           ^self osErrorIn: #WriteFile ]

 

Manfred.

 

Von: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]] Im Auftrag von Chantal Thibodeau
Gesendet: Montag, 13. Februar 2012 19:10
An: [hidden email]
Betreff: Re: File size limit

 

Thanks, I'm looking into that (the first modification wasn't enough).

 

I'm using a <nextLine> ou <upTo:> to parse the file.

 

I will get back to you later this afternoon.

 

And thanks again

----- Original Message -----

Sent: Monday, February 13, 2012 12:15 PM

Subject: Re: File size limit

 

More info.

 

There are two additional methods you should also change:

 

FileHandle >> #endByte

| large success |

large := WinLargeInteger new.

success := KernelLibrary getFileSize: self asParameter fileSizeHigh: large.

success ifFalse: [self osError].

^large asUnsignedInteger

 

KernelDLL >> #getFileSize: aFileHandle fileSizeHigh: aWinLargeInteger

<api: GetFileSizeEx ulong struct boolean>

^self invalidArgument

 

On Mon, Feb 13, 2012 at 1:21 PM, Leandro Caniglia <[hidden email]> wrote:

Be careful. I haven't tried the hint myself yet.

 

On Mon, Feb 13, 2012 at 1:06 PM, Chantal Thibodeau <[hidden email]> wrote:

Thanks, i'll wait for an answer

----- Original Message -----

Sent: Monday, February 13, 2012 10:07 AM

Subject: Re: File size limit

 

Hi!

 

The Win32 API does support 64 bit file pointers.

Currently the methods calling the API use zero for the upper 32 bit. This limits the files to 4GB.

I think it should be possible to enhance the File related classes to support the 64 bit file pointers.

See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365541(v=vs.85).aspx

 

Perhaps somebody already did this and is willing to share it?

 

Regards,

   Andreas

 

Andreas Rosenberg | eMail: [hidden email]
APIS GmbH         | Phone: +49 9482 9415-0
Im Haslet 42      | Fax: +49 9482 9415-55
93086 Wörth/D     | WWW: <http://www.apis.de/>

Germany           | <http://www.fmea.de/>

 

-----Original Message-----
From: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]]On Behalf Of Chantal Thibodeau
Sent: Montag, 13. Februar 2012 14:55
To: [hidden email]
Subject: File size limit

Hi

 

We are using VSE 3.0 for over 15 years and in this period, files are getting bigger and bigger.  We are unable to read files larger than 4 gigs.  At writing, there is no problem.

 

I know this is because of the "maxInteger" value.

 

Is there a way to bypass that limit (except splitting larger files in two) ?

 

Thanks

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management *** *** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

 

 

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***



Treffen Sie AEB vom 13.-15. März 2012 auf der LogiMAT in Stuttgart. Halle 7, Stand 261.
Vereinbaren Sie jetzt einen Termin und erhalten eine Eintrittskarte: www.aeb.de/logimat
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

AW: File size limit

Möbus. Manfred

Hi Leandro,

don’t know, guess we just scanned the class for methods using file pointers, and so fixed it.

 

Manfred

 

Von: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]] Im Auftrag von Leandro Caniglia
Gesendet: Dienstag, 14. Februar 2012 02:11
An: [hidden email]
Betreff: Re: File size limit

 

Thanks Manfred!

 

I liked your catch of FileHandle>>#position and have a question: Is this method ever used?

 

/Leandro



Treffen Sie AEB vom 13.-15. März 2012 auf der LogiMAT in Stuttgart. Halle 7, Stand 261.
Vereinbaren Sie jetzt einen Termin und erhalten eine Eintrittskarte: www.aeb.de/logimat
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Chantal Thibodeau
Thank to you all guys.
 
I tried yesterday, with an additional modificaiton to I don't remember wich method, but without your last methods.
 
I was able to read up to about 8 gigs, but no more.  I will try later today with your last modifications.  But 8 gigs is already far better than 4 gigs.
 
I'll keep you informed.
----- Original Message -----
Sent: Tuesday, February 14, 2012 2:06 AM
Subject: AW: File size limit

Hi Leandro,

don’t know, guess we just scanned the class for methods using file pointers, and so fixed it.

 

Manfred

 

Von: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]] Im Auftrag von Leandro Caniglia
Gesendet: Dienstag, 14. Februar 2012 02:11
An: [hidden email]
Betreff: Re: File size limit

 

Thanks Manfred!

 

I liked your catch of FileHandle>>#position and have a question: Is this method ever used?

 

/Leandro



Treffen Sie AEB vom 13.-15. März 2012 auf der LogiMAT in Stuttgart. Halle 7, Stand 261.
Vereinbaren Sie jetzt einen Termin und erhalten eine Eintrittskarte: www.aeb.de/logimat
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Leandro Caniglia
Chantal,

I've tested the code (mine + Manfred's) with 4, 8 and 12 GB and it worked just fine.

/Leandro


On Tue, Feb 14, 2012 at 9:15 AM, Chantal Thibodeau <[hidden email]> wrote:
Thank to you all guys.
 
I tried yesterday, with an additional modificaiton to I don't remember wich method, but without your last methods.
 
I was able to read up to about 8 gigs, but no more.  I will try later today with your last modifications.  But 8 gigs is already far better than 4 gigs.
 
I'll keep you informed.
----- Original Message -----
Sent: Tuesday, February 14, 2012 2:06 AM
Subject: AW: File size limit

Hi Leandro,

don’t know, guess we just scanned the class for methods using file pointers, and so fixed it.

 

Manfred

 

Von: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]] Im Auftrag von Leandro Caniglia
Gesendet: Dienstag, 14. Februar 2012 02:11
An: [hidden email]
Betreff: Re: File size limit

 

Thanks Manfred!

 

I liked your catch of FileHandle>>#position and have a question: Is this method ever used?

 

/Leandro



Treffen Sie AEB vom 13.-15. März 2012 auf der LogiMAT in Stuttgart. Halle 7, Stand 261.
Vereinbaren Sie jetzt einen Termin und erhalten eine Eintrittskarte: www.aeb.de/logimat
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Leandro Caniglia
Here is the (approximate) code of my test

testLargeFile

| filename long file a b pos |

filename := 'largefile.bin'.

long := (ExternalLong fromInteger: -1) asUnsignedInteger * 3. "12 GB"

file := File newFile: filename.

[

file

position: long;

nextPut: $a;

nextPut: $b]

ensure: [file close].

self assert: (File fromPath: filename) size = (long + 2).

file := File pathNameReadOnly: filename.

[

file position: long.

a := file next.

b := file next.

pos := file file handle position]

ensure: [

file close.

File remove: filename].

self

assert: a = $a;

assert: b = $b;

assert: pos = (long + 2)



On Tue, Feb 14, 2012 at 9:52 AM, Leandro Caniglia <[hidden email]> wrote:
Chantal,

I've tested the code (mine + Manfred's) with 4, 8 and 12 GB and it worked just fine.

/Leandro


On Tue, Feb 14, 2012 at 9:15 AM, Chantal Thibodeau <[hidden email]> wrote:
Thank to you all guys.
 
I tried yesterday, with an additional modificaiton to I don't remember wich method, but without your last methods.
 
I was able to read up to about 8 gigs, but no more.  I will try later today with your last modifications.  But 8 gigs is already far better than 4 gigs.
 
I'll keep you informed.
----- Original Message -----
Sent: Tuesday, February 14, 2012 2:06 AM
Subject: AW: File size limit

Hi Leandro,

don’t know, guess we just scanned the class for methods using file pointers, and so fixed it.

 

Manfred

 

Von: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]] Im Auftrag von Leandro Caniglia
Gesendet: Dienstag, 14. Februar 2012 02:11
An: [hidden email]
Betreff: Re: File size limit

 

Thanks Manfred!

 

I liked your catch of FileHandle>>#position and have a question: Is this method ever used?

 

/Leandro



Treffen Sie AEB vom 13.-15. März 2012 auf der LogiMAT in Stuttgart. Halle 7, Stand 261.
Vereinbaren Sie jetzt einen Termin und erhalten eine Eintrittskarte: www.aeb.de/logimat
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***


*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Chantal Thibodeau
In reply to this post by Leandro Caniglia
Thanks, i'm trilled to test it
----- Original Message -----
Sent: Tuesday, February 14, 2012 7:52 AM
Subject: Re: File size limit

Chantal,

I've tested the code (mine + Manfred's) with 4, 8 and 12 GB and it worked just fine.

/Leandro


On Tue, Feb 14, 2012 at 9:15 AM, Chantal Thibodeau <[hidden email]> wrote:
Thank to you all guys.
 
I tried yesterday, with an additional modificaiton to I don't remember wich method, but without your last methods.
 
I was able to read up to about 8 gigs, but no more.  I will try later today with your last modifications.  But 8 gigs is already far better than 4 gigs.
 
I'll keep you informed.
----- Original Message -----
Sent: Tuesday, February 14, 2012 2:06 AM
Subject: AW: File size limit

Hi Leandro,

don’t know, guess we just scanned the class for methods using file pointers, and so fixed it.

 

Manfred

 

Von: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]] Im Auftrag von Leandro Caniglia
Gesendet: Dienstag, 14. Februar 2012 02:11
An: [hidden email]
Betreff: Re: File size limit

 

Thanks Manfred!

 

I liked your catch of FileHandle>>#position and have a question: Is this method ever used?

 

/Leandro



Treffen Sie AEB vom 13.-15. März 2012 auf der LogiMAT in Stuttgart. Halle 7, Stand 261.
Vereinbaren Sie jetzt einen Termin und erhalten eine Eintrittskarte: www.aeb.de/logimat
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Chantal Thibodeau
In reply to this post by Leandro Caniglia
Happy Valentine's Day to you all, it works, it works !
 
I was using just a <nextLine> all the way through the file and I reach the last line.
 
Chantal
----- Original Message -----
Sent: Tuesday, February 14, 2012 7:57 AM
Subject: Re: File size limit

Here is the (approximate) code of my test

testLargeFile

| filename long file a b pos |

filename := 'largefile.bin'.

long := (ExternalLong fromInteger: -1) asUnsignedInteger * 3. "12 GB"

file := File newFile: filename.

[

file

position: long;

nextPut: $a;

nextPut: $b]

ensure: [file close].

self assert: (File fromPath: filename) size = (long + 2).

file := File pathNameReadOnly: filename.

[

file position: long.

a := file next.

b := file next.

pos := file file handle position]

ensure: [

file close.

File remove: filename].

self

assert: a = $a;

assert: b = $b;

assert: pos = (long + 2)



On Tue, Feb 14, 2012 at 9:52 AM, Leandro Caniglia <[hidden email]> wrote:
Chantal,

I've tested the code (mine + Manfred's) with 4, 8 and 12 GB and it worked just fine.

/Leandro


On Tue, Feb 14, 2012 at 9:15 AM, Chantal Thibodeau <[hidden email]> wrote:
Thank to you all guys.
 
I tried yesterday, with an additional modificaiton to I don't remember wich method, but without your last methods.
 
I was able to read up to about 8 gigs, but no more.  I will try later today with your last modifications.  But 8 gigs is already far better than 4 gigs.
 
I'll keep you informed.
----- Original Message -----
Sent: Tuesday, February 14, 2012 2:06 AM
Subject: AW: File size limit

Hi Leandro,

don’t know, guess we just scanned the class for methods using file pointers, and so fixed it.

 

Manfred

 

Von: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]] Im Auftrag von Leandro Caniglia
Gesendet: Dienstag, 14. Februar 2012 02:11
An: [hidden email]
Betreff: Re: File size limit

 

Thanks Manfred!

 

I liked your catch of FileHandle>>#position and have a question: Is this method ever used?

 

/Leandro



Treffen Sie AEB vom 13.-15. März 2012 auf der LogiMAT in Stuttgart. Halle 7, Stand 261.
Vereinbaren Sie jetzt einen Termin und erhalten eine Eintrittskarte: www.aeb.de/logimat
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***


*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: File size limit

Leandro Caniglia
Eureka!

On Tue, Feb 14, 2012 at 11:06 AM, Chantal Thibodeau <[hidden email]> wrote:
Happy Valentine's Day to you all, it works, it works !
 
I was using just a <nextLine> all the way through the file and I reach the last line.
 
Chantal
----- Original Message -----
Sent: Tuesday, February 14, 2012 7:57 AM
Subject: Re: File size limit

Here is the (approximate) code of my test

testLargeFile

| filename long file a b pos |

filename := 'largefile.bin'.

long := (ExternalLong fromInteger: -1) asUnsignedInteger * 3. "12 GB"

file := File newFile: filename.

[

file

position: long;

nextPut: $a;

nextPut: $b]

ensure: [file close].

self assert: (File fromPath: filename) size = (long + 2).

file := File pathNameReadOnly: filename.

[

file position: long.

a := file next.

b := file next.

pos := file file handle position]

ensure: [

file close.

File remove: filename].

self

assert: a = $a;

assert: b = $b;

assert: pos = (long + 2)



On Tue, Feb 14, 2012 at 9:52 AM, Leandro Caniglia <[hidden email]> wrote:
Chantal,

I've tested the code (mine + Manfred's) with 4, 8 and 12 GB and it worked just fine.

/Leandro


On Tue, Feb 14, 2012 at 9:15 AM, Chantal Thibodeau <[hidden email]> wrote:
Thank to you all guys.
 
I tried yesterday, with an additional modificaiton to I don't remember wich method, but without your last methods.
 
I was able to read up to about 8 gigs, but no more.  I will try later today with your last modifications.  But 8 gigs is already far better than 4 gigs.
 
I'll keep you informed.
----- Original Message -----
Sent: Tuesday, February 14, 2012 2:06 AM
Subject: AW: File size limit

Hi Leandro,

don’t know, guess we just scanned the class for methods using file pointers, and so fixed it.

 

Manfred

 

Von: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]] Im Auftrag von Leandro Caniglia
Gesendet: Dienstag, 14. Februar 2012 02:11
An: [hidden email]
Betreff: Re: File size limit

 

Thanks Manfred!

 

I liked your catch of FileHandle>>#position and have a question: Is this method ever used?

 

/Leandro



Treffen Sie AEB vom 13.-15. März 2012 auf der LogiMAT in Stuttgart. Halle 7, Stand 261.
Vereinbaren Sie jetzt einen Termin und erhalten eine Eintrittskarte: www.aeb.de/logimat
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***


*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

UTF-8

Chantal Thibodeau
In reply to this post by Leandro Caniglia
Hi everybody
 
Has someone already tried to read UTF-8 strings ?
 
My problem is we receive UTF-8 XML files, that are then put in a MySQL database, in UTF8 format.
 
When I read them, all the accents are screwed up.  I looked on the net but I only find code for Squeak or other, and I am unable to make it works.
 
Thanks
 
 
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: UTF-8

Sebastian Calvo
Hi:

Download
http://www.infoil.com.ar/seaside/downloads/Seaside%203.0%20rc3%202010-NOV-06.zip

Inside the zip file there is a folder "US" containing Dolphin
Smalltalk source code to handle UTF-8 strings. Maybe you can port the
code to VS. The implementation is based on standard windows api and
standard Smalltalk so I guess is an easy job :)

Best Regards
  Sebastian Calvo

2012/3/20 Chantal Thibodeau <[hidden email]>:

> Hi everybody
>
> Has someone already tried to read UTF-8 strings ?
>
> My problem is we receive UTF-8 XML files, that are then put in a MySQL
> database, in UTF8 format.
>
> When I read them, all the accents are screwed up.  I looked on the net but I
> only find code for Squeak or other, and I am unable to make it works.
>
> Thanks
>
>
> *** this signature added by listserv *** *** Visit
> http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing
> and VSWE-L membership management ***

***           this signature added by listserv             ***
*** Visit  http://www.listserv.dfn.de/archives/vswe-l.html ***
*** for archive browsing and VSWE-L membership management  ***
Reply | Threaded
Open this post in threaded view
|

Re: UTF-8

Todor Todorov
In reply to this post by Chantal Thibodeau

Use MultiByteToWideChar with CP_UTF8 to convert the UTF-8 string to Unicode and then use WideCharToMultiByte to convert the unicode string the the codepage of your SQL server (or leave it in unicode if you use the unicode fields in the database)

 

 

From: Using Visual Smalltalk for Windows/Enterprise [mailto:[hidden email]] On Behalf Of Chantal Thibodeau
Sent: 20. marts 2012 20:25
To: [hidden email]
Subject: UTF-8

 

Hi everybody

 

Has someone already tried to read UTF-8 strings ?

 

My problem is we receive UTF-8 XML files, that are then put in a MySQL database, in UTF8 format.

 

When I read them, all the accents are screwed up.  I looked on the net but I only find code for Squeak or other, and I am unable to make it works.

 

Thanks

 

 

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***

*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
Reply | Threaded
Open this post in threaded view
|

Re: UTF-8

Jon Raiford
In reply to this post by Chantal Thibodeau
If the accents are your only problem, it sounds as if something is translating the text for you.  Make sure you have configured MySQL to accept and hold unicode data.  If not, the database itself will convert the text to latin and the accents are lost.  This link might help: http://dev.hubspot.com/bid/7049/MySQL-and-Unicode-Three-Gotchas

Also, since Windows 2000, Windows APIs use UTF16 (previously used UCS2), so as Todor pointed out you may need to convert your UTF8 to UTF16 if you want to call functions that expect Unicode parameters.

Jon




From:        Chantal Thibodeau <[hidden email]>
To:        [hidden email],
Date:        03/20/2012 03:54 PM
Subject:        UTF-8
Sent by:        Using Visual Smalltalk for Windows/Enterprise <[hidden email]>




Hi everybody
 
Has someone already tried to read UTF-8 strings ?
 
My problem is we receive UTF-8 XML files, that are then put in a MySQL database, in UTF8 format.
 
When I read them, all the accents are screwed up.  I looked on the net but I only find code for Squeak or other, and I am unable to make it works.
 
Thanks
 
 
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
*** this signature added by listserv *** *** Visit http://www.listserv.dfn.de/archives/vswe-l.html *** *** for archive browsing and VSWE-L membership management ***
12