Smalltalk › Squeak › Squeak - Dev

Copying the sources file

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

9 messages Options

Andreas.Raab

Copying the sources file

Hi -

I was just trying to implement the code for condensing the changes to
the end of the sources file and ran into some 'interesting' issues. I
would expect this to work:

fullName := 'test1.sources'.
"Copy sources file; change file name accordingly"
FileStream forceNewFileNamed: fullName do:[:newFile|
sourcesFile := SourceFiles at: 1.
sourcesFile position: 0.
FileDirectory default copyFile: sourcesFile toFile: newFile.
newFile position = sourcesFile size ifFalse:[self error: 'File copy
failed'].
].

But it fails. Then I tried something else:

fullName := 'test2.sources'.
"Copy sources file; change file name accordingly"
FileStream forceNewFileNamed: fullName do:[:newFile|
sourcesFile := SourceFiles at: 1.
sourcesFile position: 0.
[sourcesFile atEnd] whileFalse:[newFile nextChunkPut: sourcesFile
nextChunk].
newFile position = sourcesFile size ifFalse:[self error: 'File copy
failed'].
].

That fails to. Then I tried:

fullName := 'test3.sources'.
"Copy sources file; change file name accordingly"
FileStream forceNewFileNamed: fullName do:[:newFile|
sourcesFile := SourceFiles at: 1.
sourcesFile position: 0.
[sourcesFile atEnd] whileFalse:[newFile nextChunkPutWithStyle:
sourcesFile nextChunkText].
newFile position = sourcesFile size ifFalse:[self error: 'File copy
failed'].
].

That fails as well. Help! What am I doing wrong? How does one copy the
sources file properly and why do the above methods not work?

Cheers,
- Andreas

johnmci

Re: Copying the sources file

On 2010-03-31, at 12:08 AM, Andreas Raab wrote:

> That fails as well. Help! What am I doing wrong? How does one copy the sources file properly and why do the above methods not work?
>
> Cheers,
> - Andreas

Well both streams are those funky
MultiByteFileStream

Soooo.

(a) your UTF8 data in the SourceFiles at: 1 is bad?
(b) your MultiByteFileStream reader code is bad?
(c) your MultiByteFileStream writer code is bad?
(d) newFile position = sourcesFile size isn't what you think it is/does...

Or combinations of above.

Likely you want to run some nice text compare tool on the two files outside of squeak and see how
sane things are.

--
===========================================================================
John M. McIntosh <[hidden email]> Twitter: squeaker68882
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
===========================================================================

smime.p7s (3K) Download Attachment

Levente Uzonyi-2

Re: Copying the sources file

In reply to this post by Andreas.Raab

On Wed, 31 Mar 2010, Andreas Raab wrote:

> Hi -
>
> I was just trying to implement the code for condensing the changes to the end
> of the sources file and ran into some 'interesting' issues. I would expect
> this to work:
>
> fullName := 'test1.sources'.
> "Copy sources file; change file name accordingly"
> FileStream forceNewFileNamed: fullName do:[:newFile|
> sourcesFile := SourceFiles at: 1.
> sourcesFile position: 0.
> FileDirectory default copyFile: sourcesFile toFile: newFile.
> newFile position = sourcesFile size ifFalse:[self error: 'File copy
> failed'].
> ].

This one is definitely a bug. Using a readOnlyCopy instead of the original
sources file makes debugging a bit easier. :)

>
> But it fails. Then I tried something else:
>
> fullName := 'test2.sources'.
> "Copy sources file; change file name accordingly"
> FileStream forceNewFileNamed: fullName do:[:newFile|
> sourcesFile := SourceFiles at: 1.
> sourcesFile position: 0.
> [sourcesFile atEnd] whileFalse:[newFile nextChunkPut: sourcesFile
> nextChunk].
> newFile position = sourcesFile size ifFalse:[self error: 'File copy
> failed'].
> ].
>
> That fails to. Then I tried:
>
> fullName := 'test3.sources'.
> "Copy sources file; change file name accordingly"
> FileStream forceNewFileNamed: fullName do:[:newFile|
> sourcesFile := SourceFiles at: 1.
> sourcesFile position: 0.
> [sourcesFile atEnd] whileFalse:[newFile nextChunkPutWithStyle:
> sourcesFile nextChunkText].
> newFile position = sourcesFile size ifFalse:[self error: 'File copy
> failed'].
> ].
>
> That fails as well. Help! What am I doing wrong? How does one copy the
> sources file properly and why do the above methods not work?

The latter two will probably fail, since the chunk reading code throws
away unnecessary whitespaces.

I checked these in 4.0 and all three are failing the same way as in 4.1.

Levente

>
> Cheers,
> - Andreas
>
>

Levente Uzonyi-2

Re: Copying the sources file

On Wed, 31 Mar 2010, Levente Uzonyi wrote:

> On Wed, 31 Mar 2010, Andreas Raab wrote:
>
>> Hi -
>>
>> I was just trying to implement the code for condensing the changes to the
>> end of the sources file and ran into some 'interesting' issues. I would
>> expect this to work:
>>
>> fullName := 'test1.sources'.
>> "Copy sources file; change file name accordingly"
>> FileStream forceNewFileNamed: fullName do:[:newFile|
>> sourcesFile := SourceFiles at: 1.
>> sourcesFile position: 0.
>> FileDirectory default copyFile: sourcesFile toFile: newFile.
>> newFile position = sourcesFile size ifFalse:[self error: 'File copy
>> failed'].
>> ].
>
> This one is definitely a bug. Using a readOnlyCopy instead of the original
> sources file makes debugging a bit easier. :)

Okay, found the cause of the issue. MultiByteFileStream doesn't implement
it's own version of #nextInto: which is used by #copyFile:toFile:,
therefore the file is not decoded. Replacing #nextInto: with an
appropriate #next: send passes the test.

Implementing MultiByteFileStream >> #nextInto: is a bit tricky, since if
the argument is a ByteString, then there's a chance that the result cannot
be stored into the buffer. In similar cases the ByteString is swapped with
a WideString, but in this case it can become a source of bugs, since the
sender of #nextInto: may expect that the buffer is still a ByteString.

IMHO #copyFile:toFile: shouldn't exist, it's just a utility method,
totally unrelated to FileDirectory. It also assumes that the files are in
"ascii" mode which is wrong.

Levente

>
>>
>> But it fails. Then I tried something else:
>>
>> fullName := 'test2.sources'.
>> "Copy sources file; change file name accordingly"
>> FileStream forceNewFileNamed: fullName do:[:newFile|
>> sourcesFile := SourceFiles at: 1.
>> sourcesFile position: 0.
>> [sourcesFile atEnd] whileFalse:[newFile nextChunkPut: sourcesFile
>> nextChunk].
>> newFile position = sourcesFile size ifFalse:[self error: 'File copy
>> failed'].
>> ].
>>
>> That fails to. Then I tried:
>>
>> fullName := 'test3.sources'.
>> "Copy sources file; change file name accordingly"
>> FileStream forceNewFileNamed: fullName do:[:newFile|
>> sourcesFile := SourceFiles at: 1.
>> sourcesFile position: 0.
>> [sourcesFile atEnd] whileFalse:[newFile nextChunkPutWithStyle:
>> sourcesFile nextChunkText].
>> newFile position = sourcesFile size ifFalse:[self error: 'File copy
>> failed'].
>> ].
>>
>> That fails as well. Help! What am I doing wrong? How does one copy the
>> sources file properly and why do the above methods not work?
>
> The latter two will probably fail, since the chunk reading code throws away
> unnecessary whitespaces.
>
> I checked these in 4.0 and all three are failing the same way as in 4.1.
>
>
> Levente
>
>>
>> Cheers,
>> - Andreas
>>
>>
>
>

johnmci

Re: Copying the sources file

On 2010-03-31, at 3:59 PM, Levente Uzonyi wrote:

> IMHO #copyFile:toFile: shouldn't exist, it's just a utility method, totally unrelated to FileDirectory. It also assumes that the files are in "ascii" mode which is wrong.
>
>
> Levente

+1 In Sophie we replaced this with a primitive to which had the responsibility to ask the operating system to copy the files.
The complexity of meta-data (security, dates/times, Access lists, stuff) on a *file* make Squeak's desire to copy just the bits rather 1970 ish...
Oh wait pre that since it doesn't preserve the from file security settings either...

--
===========================================================================
John M. McIntosh <[hidden email]> Twitter: squeaker68882
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
===========================================================================

smime.p7s (3K) Download Attachment

Levente Uzonyi-2

Re: Copying the sources file

On Wed, 31 Mar 2010, John M McIntosh wrote:

>
> On 2010-03-31, at 3:59 PM, Levente Uzonyi wrote:
>
>> IMHO #copyFile:toFile: shouldn't exist, it's just a utility method, totally unrelated to FileDirectory. It also assumes that the files are in "ascii" mode which is wrong.
>>
>>
>> Levente
>
>
> +1 In Sophie we replaced this with a primitive to which had the responsibility to ask the operating system to copy the files.
> The complexity of meta-data (security, dates/times, Access lists, stuff) on a *file* make Squeak's desire to copy just the bits rather 1970 ish...
> Oh wait pre that since it doesn't preserve the from file security settings either...

What's really "funny" about this method is that it doesn't really copy
files, just the copies the contents of a ReadStream to WriteStream using
a ByteString as a buffer. It doesn't use files at all.

Levente

>
>
> --
> ===========================================================================
> John M. McIntosh <[hidden email]> Twitter: squeaker68882
> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
> ===========================================================================
>
>
>
>
>

johnmci

Re: Copying the sources file

On 2010-03-31, at 5:59 PM, Levente Uzonyi wrote:
>> +1 In Sophie we replaced this with a primitive to which had the responsibility to ask the operating system to copy the files.
>> The complexity of meta-data (security, dates/times, Access lists, stuff) on a *file* make Squeak's desire to copy just the bits rather 1970 ish...
>> Oh wait pre that since it doesn't preserve the from file security settings either...
>
> What's really "funny" about this method is that it doesn't really copy files, just the copies the contents of a ReadStream to WriteStream using a ByteString as a buffer. It doesn't use files at all.
>
>
> Levente

Ya, in Sophie everything was a URI. So in the copyFromURI:toURI we would check and say oh look
both URI's are file based, so do this copyFile:to: primitive. Otherwise the URI read/write logic came into play...

--
===========================================================================
John M. McIntosh <[hidden email]> Twitter: squeaker68882
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
===========================================================================

smime.p7s (3K) Download Attachment

Andreas.Raab

Re: Copying the sources file

In reply to this post by Levente Uzonyi-2

On 3/31/2010 3:59 PM, Levente Uzonyi wrote:
> Okay, found the cause of the issue. MultiByteFileStream doesn't
> implement it's own version of #nextInto: which is used by
> #copyFile:toFile:, therefore the file is not decoded. Replacing
> #nextInto: with an appropriate #next: send passes the test.

Thanks for the help, I've worked around the problem by using #next: and
#nextPutAll: which seems to work. It's yucky though; if I hadn't added
my sanity test I wouldn't even have found the problem and the sources
file would've been silently broken.

Cheers,
- Andreas

> Implementing MultiByteFileStream >> #nextInto: is a bit tricky, since if
> the argument is a ByteString, then there's a chance that the result
> cannot be stored into the buffer. In similar cases the ByteString is
> swapped with a WideString, but in this case it can become a source of
> bugs, since the sender of #nextInto: may expect that the buffer is still
> a ByteString.
>
> IMHO #copyFile:toFile: shouldn't exist, it's just a utility method,
> totally unrelated to FileDirectory. It also assumes that the files are
> in "ascii" mode which is wrong.
>
>
> Levente
>
>>
>>>
>>> But it fails. Then I tried something else:
>>>
>>> fullName := 'test2.sources'.
>>> "Copy sources file; change file name accordingly"
>>> FileStream forceNewFileNamed: fullName do:[:newFile|
>>> sourcesFile := SourceFiles at: 1.
>>> sourcesFile position: 0.
>>> [sourcesFile atEnd] whileFalse:[newFile nextChunkPut: sourcesFile
>>> nextChunk].
>>> newFile position = sourcesFile size ifFalse:[self error: 'File copy
>>> failed'].
>>> ].
>>>
>>> That fails to. Then I tried:
>>>
>>> fullName := 'test3.sources'.
>>> "Copy sources file; change file name accordingly"
>>> FileStream forceNewFileNamed: fullName do:[:newFile|
>>> sourcesFile := SourceFiles at: 1.
>>> sourcesFile position: 0.
>>> [sourcesFile atEnd] whileFalse:[newFile nextChunkPutWithStyle:
>>> sourcesFile nextChunkText].
>>> newFile position = sourcesFile size ifFalse:[self error: 'File copy
>>> failed'].
>>> ].
>>>
>>> That fails as well. Help! What am I doing wrong? How does one copy
>>> the sources file properly and why do the above methods not work?
>>
>> The latter two will probably fail, since the chunk reading code throws
>> away unnecessary whitespaces.
>>
>> I checked these in 4.0 and all three are failing the same way as in 4.1.
>>
>>
>> Levente
>>
>>>
>>> Cheers,
>>> - Andreas
>>>
>>>
>>
>>
>
>

Chris Muller-3

Re: Copying the sources file

In reply to this post by johnmci

> The complexity of meta-data (security, dates/times, Access lists, stuff) on a *file* make Squeak's desire to copy just the bits rather 1970 ish...

Well, you could also say, it does it in a way that is
platform-independent-ish... Lowest-common denominator, but the
default unix cp command loses time-stamp and permissions as well
unless you use -p...

Levente wrote:
> What's really "funny" about this method is that it doesn't really copy files, just the copies the contents of a ReadStream to WriteStream using a ByteString as a buffer. It doesn't use files at all.

It is a "utility" method because it is called by API methods on
FileDirectory. So perhaps #copyFile:toFile: should just be made
private and then you can think of it as merely a composed-method.

> Oh wait pre that since it doesn't preserve the from file security settings either...
>
>
> --
> ===========================================================================
> John M. McIntosh <[hidden email]> Twitter: squeaker68882
> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
> ===========================================================================
>
>
>
>
>
>
>
>