Re: [Pharo-users] Trouble opening large files

Hi Nicolai,

On Tue, Oct 14, 2014 at 11:26 AM, Nicolai Hess <[hidden email]> wrote:

There is a bug report on mantis for squeaks unix vm.

I think this applies to pharo too, although I don't know if this

bug is still valid on recent squeak vm.

http://bugs.squeak.org/view.php?id=7522

yes, one must compile with -D_FILE_OFFSET_BITS=64.  The Cog VMs are also built with -D_GNU_SOURCE.

Here's a line from a Squeak file list on the current Cog VM:

(2014.10.11 07:00:46 7,115,143,880) Formula1.2014.Round16.Russia.Qualifying.BBCOneHD.1080i.H264.English-wserhkzt.ts 

No 32-bit limit here.

2014-10-14 17:43 GMT+02:00 Evan Donahue <[hidden email]>:

The OS is Arch Linux.

I can read the file with less.

The problem, insofar as I can trace it, seems to stem from this line in UnixStore:

Primitives lookupDirectory: encodedPath filename: encodedBasename

When I have my 57G file there this line returns nil. If I move the 57G file and create a small file with the same name, the same command successfully finds the file. I am not sure how large a file must be to cause this issue, but A 1.5G file works fine.

On Tue, Oct 14, 2014 at 8:17 AM, Nicolai Hess <[hidden email]> wrote:

2014-10-14 6:38 GMT+02:00 Evan Donahue <[hidden email]>:

Hi, thanks for the reply.

The response is the same: "MessageNotUnderstood: False>>humanReadableSIByteSize."

This happens both to print-it as well as to do-it-and-go. Running the command on the neighboring "wiki.torrent" torrent file yields the correct 54kb.

Thanks,
Evan

On Tue, Oct 14, 2014 at 12:24 AM, Tudor Girba <[hidden email]> wrote:

Hi,

If I understand correctly, the failure occurs while navigating in the "Items" presentation.

I cannot reproduce this problem because I do not have enough disk space for such a large file :). But, could you do the following and let me know what the outcome is:

'path/to/your/large/file.xml' asFileReference humanReadableSize

?

Cheers,

Doru

On Tue, Oct 14, 2014 at 2:27 AM, Evan Donahue <[hidden email]> wrote:

Hello, I've run into some odd behavior and wanted to check whether I might be missing something:

I have downloaded a copy of the english wikipedia as an xml file and am hoping to (sax) parse it. However, I can't even seem to get pharo to recognize that the file exists.

If I open FileSystem disk root in the playground and naigate, attempting to enter the folder containing the (57G) xml file fails with "MessageNotUnderstood: False>>humanReadableSIByteSize." Likewise if I get a FileReference with FileSystem disk root / 'path' / 'to' / 'file' then self exists returns false and the parser fails.

Am I doing something wrong? Should I be able to do this?

The version number is #40283

Thanks,

Evan

--

www.tudorgirba.com

"Every thing has its own flow"

Which OS ?

Can you check with other programs if this file is readable at all?

Nicolai

--
best,

Eliot

Do we create an issue for the Pharo-vm?

Stef

On 14/10/14 21:29, Nicolai Hess wrote:

Ah, good to know. At least for squeak,
bug 7522 can be closed :)

Nicolai

2014-10-14 20:49 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

On Tue, Oct 14, 2014 at 11:26 AM, Nicolai Hess <[hidden email]> wrote:

There is a bug report on mantis for squeaks unix vm.

I think this applies to pharo too, although I don't know if this

bug is still valid on recent squeak vm.

http://bugs.squeak.org/view.php?id=7522

yes, one must compile with -D_FILE_OFFSET_BITS=64.  The Cog VMs are also built with -D_GNU_SOURCE.

Here's a line from a Squeak file list on the current Cog VM:

(2014.10.11 07:00:46 7,115,143,880) Formula1.2014.Round16.Russia.Qualifying.BBCOneHD.1080i.H264.English-wserhkzt.ts 

No 32-bit limit here.

2014-10-14 17:43 GMT+02:00 Evan Donahue <[hidden email]>:

The OS is Arch Linux.

I can read the file with less.

The problem, insofar as I can trace it, seems to stem from this line in UnixStore:

Primitives lookupDirectory: encodedPath filename: encodedBasename

When I have my 57G file there this line returns nil. If I move the 57G file and create a small file with the same name, the same command successfully finds the file. I am not sure how large a file must be to cause this issue, but A 1.5G file works fine.

On Tue, Oct 14, 2014 at 8:17 AM, Nicolai Hess <[hidden email]> wrote:

2014-10-14 6:38 GMT+02:00 Evan Donahue <[hidden email]>:

Hi, thanks for the reply.

The response is the same: "MessageNotUnderstood: False>>humanReadableSIByteSize."

This happens both to print-it as well as to do-it-and-go. Running the command on the neighboring "wiki.torrent" torrent file yields the correct 54kb.

Thanks,
Evan

On Tue, Oct 14, 2014 at 12:24 AM, Tudor Girba <[hidden email]> wrote:

Hi,

If I understand correctly, the failure occurs while navigating in the "Items" presentation.

I cannot reproduce this problem because I do not have enough disk space for such a large file :). But, could you do the following and let me know what the outcome is:

'path/to/your/large/file.xml' asFileReference humanReadableSize

?

Cheers,

Doru

On Tue, Oct 14, 2014 at 2:27 AM, Evan Donahue <[hidden email]> wrote:

Hello, I've run into some odd behavior and wanted to check whether I might be missing something:

I have downloaded a copy of the english wikipedia as an xml file and am hoping to (sax) parse it. However, I can't even seem to get pharo to recognize that the file exists.

If I open FileSystem disk root in the playground and naigate, attempting to enter the folder containing the (57G) xml file fails with "MessageNotUnderstood: False>>humanReadableSIByteSize." Likewise if I get a FileReference with FileSystem disk root / 'path' / 'to' / 'file' then self exists returns false and the parser fails.

Am I doing something wrong? Should I be able to do this?

The version number is #40283

Thanks,

Evan

--

www.tudorgirba.com

"Every thing has its own flow"

Which OS ?

Can you check with other programs if this file is readable at all?

Nicolai

--
best,

Eliot