Smalltalk › Squeak › Squeak - Dev

Wrote a little about OnScreenKeyboardMorph

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

17 messages Options

Tony Garnock-Jones-5

Wrote a little about OnScreenKeyboardMorph

Hi all,

I finally blogged about OnScreenKeyboardMorph:
https://eighty-twenty.org/2021/04/20/on-screen-keyboard-morph

Cheers,
Tony

Douglas Brebner-2

Re: Wrote a little about OnScreenKeyboardMorph

On 20/04/2021 16:42, Tony Garnock-Jones wrote:
> Hi all,
>
> I finally blogged about OnScreenKeyboardMorph:
> https://eighty-twenty.org/2021/04/20/on-screen-keyboard-morph
>
This is very cool.

Tobias Pape

Re: Wrote a little about OnScreenKeyboardMorph

In reply to this post by Tony Garnock-Jones-5

Hey Tony

> On 20. Apr 2021, at 17:42, Tony Garnock-Jones <[hidden email]> wrote:
>
> Hi all,
>
> I finally blogged about OnScreenKeyboardMorph: https://eighty-twenty.org/2021/04/20/on-screen-keyboard-morph

This is mighty cool!

Best regards
-Tobias

Douglas Brebner-2

Re: Wrote a little about OnScreenKeyboardMorph

In reply to this post by Tony Garnock-Jones-5

On 20/04/2021 16:42, Tony Garnock-Jones wrote:
> Hi all,
>
> I finally blogged about OnScreenKeyboardMorph:
> https://eighty-twenty.org/2021/04/20/on-screen-keyboard-morph
>
> Cheers,
> Tony

I just remembered, Squeak has (used to have?) a handwriting recognition
system called Genie. That may make a useful addition to a phone UI.
Maybe even resurrect graffiti from the palm pdas :)

Tom Beckmann

Re: Wrote a little about OnScreenKeyboardMorph

Hi Douglas, hi Tony,

> I just remembered, Squeak has (used to have?) a handwriting recognition

> system called Genie. That may make a useful addition to a phone UI.

> Maybe even resurrect graffiti from the palm pdas :)

Chiming in on that note, I had experimented with that a while ago as well and it turned out pretty fun I think, here's a quick demo video [1] and the source code [2]. I believe it would be rather easy to make work if combined with your keypress generation logic, Tony.

Best,

Tom

[1] https://archive.org/details/handwriting-2021-04-21_15.44.40

[2] https://github.com/tom95/sandblocks/blob/master/packages/Sandblocks-Core/SBHandWritingInput.class.st

On Wed, Apr 21, 2021 at 3:36 PM Douglas Brebner <[hidden email]> wrote:

On 20/04/2021 16:42, Tony Garnock-Jones wrote:
> Hi all,
>
> I finally blogged about OnScreenKeyboardMorph:
> https://eighty-twenty.org/2021/04/20/on-screen-keyboard-morph
>
> Cheers,
> Tony

I just remembered, Squeak has (used to have?) a handwriting recognition
system called Genie. That may make a useful addition to a phone UI.
Maybe even resurrect graffiti from the palm pdas :)

Craig Latta

Re: Wrote a little about OnScreenKeyboardMorph

In reply to this post by Douglas Brebner-2

Hi--

Douglas Brebner wrote:

> I just remembered, Squeak has (used to have?) a handwriting
> recognition system called Genie. That may make a useful addition to a
> phone UI. Maybe even resurrect graffiti from the palm pdas :)

And on the iPad, there's Scribble. I wrote SqueakJS support for
it. I hide an invisible HTML text input element underneath wherever
you're entering text, and Scribble recognizes it. It does both ink-based
recognition and the sliding-around-on-an-onscreen-keyboard thing, which
I really like.

https://vimeo.com/494744698

-C

--
Craig Latta :: research computer scientist
Black Page Digital :: Berkeley, California
663137D7940BF5C0AFC :: 1349FB2ADA32C4D5314CE

codefrau

SqueakJS keyboard input (was: Wrote a little about OnScreenKeyboardMorph)

On Wed, Apr 21, 2021 at 1:55 PM Craig Latta <[hidden email]> wrote:

Hi--

Douglas Brebner wrote:

> I just remembered, Squeak has (used to have?) a handwriting
> recognition system called Genie. That may make a useful addition to a
> phone UI. Maybe even resurrect graffiti from the palm pdas :)

And on the iPad, there's Scribble. I wrote SqueakJS support for
it. I hide an invisible HTML text input element underneath wherever
you're entering text, and Scribble recognizes it. It does both ink-based
recognition and the sliding-around-on-an-onscreen-keyboard thing, which
I really like.

https://vimeo.com/494744698

Oh cool! Does it need image-side support? If not I'd love to get this into SqueakJS proper - I think having a hidden text field might solve the various issues we have on e.g. Android, or with copy/paste on Safari.

Vanessa

Tony Garnock-Jones-5

Re: Wrote a little about OnScreenKeyboardMorph

In reply to this post by Tom Beckmann

On 4/21/21 3:53 PM, Tom Beckmann wrote:
> Chiming in on that note, I had experimented with that a while ago as
> well and it turned out pretty fun I think, here's a quick demo video [1]
> and the source code [2]. I believe it would be rather easy to make work
> if combined with your keypress generation logic, Tony.

That's awesome! It'd make a great addition to a phone-based Smalltalk.

I've set a bookmark to come back to this once I have more of the
infrastructure worked out.

Regards,
Tony

Craig Latta

Re: SqueakJS keyboard input (was: Wrote a little about OnScreenKeyboardMorph)

In reply to this post by codefrau

I wrote:

> ...on the iPad, there's Scribble. I wrote SqueakJS support for it. I
> hide an invisible HTML text input element underneath wherever you're
> entering text, and Scribble recognizes it. It does both ink-based
> recognition and the sliding-around-on-an-onscreen-keyboard thing,
> which I really like.
>
> https://vimeo.com/494744698 <https://vimeo.com/494744698>

Vanessa responds:

> Oh cool! Does it need image-side support? If not I'd love to get this
> into SqueakJS proper - I think having a hidden text field might solve
> the various issues we have on e.g. Android, or with copy/paste on
> Safari.

Yeah! This use case involves a tiny bit of image-side support, so
that the hidden recognizer is moved properly whenever a Smalltalk window
is activated. I'll submit a pull request for the rest of it.

-C

--
Craig Latta :: research computer scientist
Black Page Digital :: Berkeley, California
663137D7940BF5C0AFC :: 1349FB2ADA32C4D5314CE

codefrau

Re: SqueakJS keyboard input (was: Wrote a little about OnScreenKeyboardMorph)

On Thu, Apr 22, 2021 at 1:13 PM Craig Latta <[hidden email]> wrote:

I wrote:

> ...on the iPad, there's Scribble. I wrote SqueakJS support for it. I
> hide an invisible HTML text input element underneath wherever you're
> entering text, and Scribble recognizes it. It does both ink-based
> recognition and the sliding-around-on-an-onscreen-keyboard thing,
> which I really like.
>
> https://vimeo.com/494744698 <https://vimeo.com/494744698>

Vanessa responds:

> Oh cool! Does it need image-side support? If not I'd love to get this
> into SqueakJS proper - I think having a hidden text field might solve
> the various issues we have on e.g. Android, or with copy/paste on
> Safari.

Yeah! This use case involves a tiny bit of image-side support, so
that the hidden recognizer is moved properly whenever a Smalltalk window
is activated. I'll submit a pull request for the rest of it.

Does it use the setCompositionWindow mechanism? I always thought we should change ImmAbstractPlatform to call a generic primSetCompositionWindowPosition in ImmPlugin, which doesn't exist right now, but would allow platforms to hook into that mechanism without image changes.

Then again, a focus / blur primitive would be good too so we can show/hide the soft keyboard automatically. It should get the bounds of the text area, and possibly the cursor position, too. With that information the VM could automatically zoom to a text field on phones ... So maybe a new SoftKeyboardPlugin?

Vanessa

Christoph Thiede

Re: Wrote a little about OnScreenKeyboardMorph

In reply to this post by Tony Garnock-Jones-5

Cool stuff! I wish we could also use this on Android. :-)

Best,

Christoph

Am 20.04.2021 17:42 schrieb Tony Garnock-Jones <[hidden email]>:

Hi all,

I finally blogged about OnScreenKeyboardMorph:
https://eighty-twenty.org/2021/04/20/on-screen-keyboard-morph

Cheers,
Tony

Carpe Squeak!

Tony Garnock-Jones-5

Re: Wrote a little about OnScreenKeyboardMorph

On 4/25/21 7:14 PM, Christoph Thiede wrote:
> Cool stuff! I wish we could also use this on Android. :-)

Me too. Smalltalk is (close to) what Android could have been... It just
needs capability-security, better image management, nested VMs... a few
minor details :-)

marcel.taeumel

Re: Wrote a little about OnScreenKeyboardMorph

> It just needs capability-security, better image management,

> nested VMs... a few minor details :-)

All these people with their feature requests ... ;-D

Best,

Marcel

Am 26.04.2021 12:54:03 schrieb Tony Garnock-Jones <[hidden email]>:
On 4/25/21 7:14 PM, Christoph Thiede wrote:
> Cool stuff! I wish we could also use this on Android. :-)

Me too. Smalltalk is (close to) what Android could have been... It just
needs capability-security, better image management, nested VMs... a few
minor details :-)

marcel.taeumel

Re: Wrote a little about OnScreenKeyboardMorph

Hey Tony,

here is a quick hack that shows how to add more space to fight that "fat finger problem". :-D

Best,

Marcel

Am 26.04.2021 17:06:42 schrieb Marcel Taeumel <[hidden email]>:
> It just needs capability-security, better image management,
> nested VMs... a few minor details :-)

All these people with their feature requests ... ;-D

Best,
Marcel

Am 26.04.2021 12:54:03 schrieb Tony Garnock-Jones <[hidden email]>:
On 4/25/21 7:14 PM, Christoph Thiede wrote:
> Cool stuff! I wish we could also use this on Android. :-)

Me too. Smalltalk is (close to) what Android could have been... It just
needs capability-security, better image management, nested VMs... a few
minor details :-)

small-screen-hacks.1.cs (10K) Download Attachment

Christoph Thiede

Re: Wrote a little about OnScreenKeyboardMorph

Hi Marcel,

except for the ' ? ' part, which is really hacky, I like the general idea.

+1 for integrating this into Trunk. :-)

(In a later step, we could even recognize the touch/tablet mode from the host system if supported - e.g. in Windows 10 or SqueakJS and automatically adjust the Squeak appearance ...)

Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Taeumel, Marcel
Gesendet: Montag, 26. April 2021 18:07:23
An: squeak-dev
Betreff: Re: [squeak-dev] Wrote a little about OnScreenKeyboardMorph

Hey Tony,

here is a quick hack that shows how to add more space to fight that "fat finger problem". :-D

Best,

Marcel

Am 26.04.2021 17:06:42 schrieb Marcel Taeumel <[hidden email]>:

> It just needs capability-security, better image management,
> nested VMs... a few minor details :-)

All these people with their feature requests ... ;-D

Best,

Marcel

Am 26.04.2021 12:54:03 schrieb Tony Garnock-Jones <[hidden email]>:

On 4/25/21 7:14 PM, Christoph Thiede wrote:
> Cool stuff! I wish we could also use this on Android. :-)

Me too. Smalltalk is (close to) what Android could have been... It just
needs capability-security, better image management, nested VMs... a few
minor details :-)

Carpe Squeak!

marcel.taeumel

Re: Wrote a little about OnScreenKeyboardMorph

Ah, no. All of it is a hack. That should not be in Trunk in this form.

From: Squeak-dev <[hidden email]> on behalf of Thiede, Christoph <[hidden email]>
Sent: Saturday, May 1, 2021 6:56:20 PM
To: squeak-dev <[hidden email]>
Subject: Re: [squeak-dev] Wrote a little about OnScreenKeyboardMorph

Hi Marcel,

except for the ' ? ' part, which is really hacky, I like the general idea.

+1 for integrating this into Trunk. :-)

(In a later step, we could even recognize the touch/tablet mode from the host system if supported - e.g. in Windows 10 or SqueakJS and automatically adjust the Squeak appearance ...)

Best,

Christoph

Hey Tony,

here is a quick hack that shows how to add more space to fight that "fat finger problem". :-D

Best,

Marcel

Am 26.04.2021 17:06:42 schrieb Marcel Taeumel <[hidden email]>:

> It just needs capability-security, better image management,
> nested VMs... a few minor details :-)

All these people with their feature requests ... ;-D

Best,

Marcel

Am 26.04.2021 12:54:03 schrieb Tony Garnock-Jones <[hidden email]>:

On 4/25/21 7:14 PM, Christoph Thiede wrote:
> Cool stuff! I wish we could also use this on Android. :-)

Me too. Smalltalk is (close to) what Android could have been... It just
needs capability-security, better image management, nested VMs... a few
minor details :-)

image.png (240K) Download Attachment

Christoph Thiede

Re: Wrote a little about OnScreenKeyboardMorph

But if we could refactor it ... :-)

Von: Squeak-dev <[hidden email]> im Auftrag von Taeumel, Marcel
Gesendet: Sonntag, 2. Mai 2021 12:54:14
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] Wrote a little about OnScreenKeyboardMorph

Ah, no. All of it is a hack. That should not be in Trunk in this form.

Hi Marcel,

except for the ' ? ' part, which is really hacky, I like the general idea.

+1 for integrating this into Trunk. :-)

(In a later step, we could even recognize the touch/tablet mode from the host system if supported - e.g. in Windows 10 or SqueakJS and automatically adjust the Squeak appearance ...)

Best,

Christoph

Hey Tony,

here is a quick hack that shows how to add more space to fight that "fat finger problem". :-D

Best,

Marcel

Am 26.04.2021 17:06:42 schrieb Marcel Taeumel <[hidden email]>:

> It just needs capability-security, better image management,
> nested VMs... a few minor details :-)

All these people with their feature requests ... ;-D

Best,

Marcel

Am 26.04.2021 12:54:03 schrieb Tony Garnock-Jones <[hidden email]>:

On 4/25/21 7:14 PM, Christoph Thiede wrote:
> Cool stuff! I wish we could also use this on Android. :-)

Me too. Smalltalk is (close to) what Android could have been... It just
needs capability-security, better image management, nested VMs... a few
minor details :-)

Carpe Squeak!