Find text in methods

Hi Andy or Blair,

I find it odd that when I search for a string, in Dolphin, using the
Methods/Browse/Containing text... in CHB, the search is case sensitive
and there are is wild card searching\

Yet with Class Find I can do both.

Is this so hard to implement?

Costas

Costas

You wrote in message news:[hidden email]...
>
> I find it odd that when I search for a string, in Dolphin, using the
> Methods/Browse/Containing text... in CHB, the search is case sensitive
> and there are is wild card searching\
>
> Yet with Class Find I can do both.
>
> Is this so hard to implement?

No, but it is hard to implement at the speed of the existing search (which
uses a very fast Boyer-Moore string searching primitive). Remember that
several megabytes of source code on disk need to be searched. We've been
meaning to do a RegEx engine with primitive support for some time which
would allow for much greater flexibility in string searchs (and be useful
for doing command line apps/CGI type stuff), but other things have been
higher priority. Of course it could be done in ST, but for speed employing
an existing C/C++ library would be preferable. Most of the time involved
would be in locating an appropriate library that fits in well, and it
doesn't have to be done as primitives - DLL calls in Dolphin are almost
immeasurably slower than true primitives, so the only reason for using the
latter is to be able to employ internal VM services.

One can do wild card searching when searching for references and
implementations of selectors in D4 too, so it isn't often necessary to do a
string search now, a fact which pushed RegEx of the list for D4.

Regards

Blair

Costas Menico wrote:

> ...
> I find it odd that when I search for a string, in Dolphin, using the
> Methods/Browse/Containing text... in CHB, the search is case sensitive
> and there are is wild card searching\

What I'd like to see is a utility to search for method names containing a
given string with wildcards, case-insensitive. Even without wildcards this
would be very useful. For instance find all methods whose name contains
'*get*' or 'get*'. In my other environment (VSE) I use it all the time
even though I also have a function to search based on strings inside the
source-code.

The latter is naturally much slower but I've implemented it as a
background task, so when the results are in, a browser pops up, and I can
have multiple searches going on simultaneously.

Panu Viljamaa

Panu,

> What I'd like to see is a utility to search for method names containing a
> given string with wildcards, case-insensitive.

You'll like Dolphin V4 then because it has exactly that capability.

Ian

Ian Bartholomew wrote:

> Panu,
>> What I'd like to see is a utility to search for method names containing a
>> given string with wildcards, case-insensitive.
>
> You'll like Dolphin V4 then because it has exactly that capability.

Great. I'm getting excited.

Seems like Dolphin has a lot of momentum behind it.
It might soon be the best Windows development platform, or is it already ?

-Panu

"panu" <[hidden email]> wrote in message news:[hidden email]...
QKS Smalltalk browsers have regex search capabilities. There is a RegEx
engine built into the AOS VM architecture. Here are a few things that might
be useful in thinking about the search pattern design.

I've detailed some of the capabilities and a snippet of the code with the
idea that it might provide some useful ideas in extending the Dolphin search
facilities.

1. Allow searching just the current class.
2. Allow searching just the current class family.
3. Allow searching just the current project (scope).
4. Allow searching the entire method space.

a) Limit search to ##constants
b) Limit search to '...' or `...' string and text literals.
c) Limit search to #selectors (messages used).
d) Limit search to #selectors (messages implemented).
e) Limit search to <<ffi's>> (foreign function calls).
f) Limit search to pool variables (global/shared variables)
g) Limit search to annotation tags (<'a tag'>)
h) Limit search to protocols (categories)

*) Allow a creation of custom filters that can be named and saved for
layered application in subsequent searches.

The regex search speed is instantaneous across the entire source base of
some 17,000 methods ~50MB of source.

The pattern parser and driver code is located in:
    CodeBrowser<<#doLookupOf: within our Win32 or Mac versions.
There are some additional methods in <Method>. The pattern tool actually
works across a wide range of elements within the repository system including
classes. But most of the time I end up using it to quickly search through
methods.
===============

The technique that makes it fast is the following:

a) The VM provides an object space walker. "Method instancesDo:" or "Method
instances do:". The former is slightly faster and more efficient but the
latter functions as controllable iterator.

b) Every method is then enumerated. Noting that the scanned method elements
are typically in RAM.

c) For each method the appropriate fields/literals are enumerated. This can
include parsing the opcodes to find intrinsic selectors.

d) All elements are pushed through the RegEx filter and any qualifying
methods are retained for displaying in a <MethodChooser> that is opened with
a drag-drop link-pipe tying its selection output to the selection input of
the current browser.

e) <Method> provides #containsPattern:mode: to actually do the filtering.
The <mode> flag array to allow searching across a combination of criteria
[a-h].

Full text searching across the repository sources is significantly slower
taking a number seconds to minutes depending on the search. This is
typically only done when refactoring large portions of the code base, such
as re-organizing documentation or v2/v3 directives.

The most common pattern I use is:
    "@g..."

Where "@" means pattern search methods.
      "g" search all methods.
      "..." is whatever symbol/selector I'm looking for.
===========================
Here is a grab of some of the code:

    (*
        @[Scope][Type][No RX]pattern
    *)

   "No RX: Check for regular expression requests"
    (c = $*)
        ifTrue: [
            Speakers beep.
            doRegEx := true. c := query next]
        ifFalse: [doRegEx := false].

   "SCOPE: Is it a global pattern request?"
    (Switch new)
    case: [c » $g] do: [scope := #global. c := query next];
    case: [c » $f] do: [scope := #family. c := query next];
    case: [c » $c] do: [scope := #class. c := query next];
    default: [scope := #global];
    on: true.

   "TYPE: Check for filtering"
    (Switch new)
    case: ['``'' includes: c] do: [op := #string. c := query next];
    case: [c = $<] do: [op := #tag. c := query next];
    case: [c = $#] do: [
        (query peekNext) = $# ifTrue:
        [
            query next.
            op := #const.
            c := query next
        ] ifFalse:
        [
            op := #selector.
            c := query next.
        ].
    ];
    case: [c = $$] do: [op := #selectorRefs. c := query next];
    case: [c = $@] do: [op := #libVarRefs. c := query next];
    case: [c = $~] do: [op := #browseCategoryRefs. c := query next];
    default: [op := #selector. c := query next];
    on: true.

   "If it is a wrapper then consume it"
    (Switch new)
    case: [c = $"] do: [pattern := query upTo: $"];
    case: [c = $"] do: [pattern := query upTo: $"];
    default: [pattern := query backup; remainingContents];
    on: true.

   "Prepare the regular expression"
    doRegEx
        ifTrue: [rx := RegEx equivalencePattern: pattern]
        ifFalse: [rx := RegEx literalEquivalencePattern: pattern].

   "Obtain a scoping query filter"
    queryFilter := [:m |
        (m containsPattern: rx mode: mode) ifTrue:
        [
            | meta result |

            meta := m methodDictionary metastructure.

           "Apply scope filters"
            result := (Switch new)
            case: #class do: [meta singleton == iClass];
            case: #family do: [meta singleton == iClass
                or: [iClass isKindOf: meta]];
            default: [true];
            on: scope.

           "Finally ensure that it is current"
            result and: [m isCurrent]
        ].
    ].

   "Prepare to dispatch on the operation"
    (Switch new)
    case: #string do:
    [
        mode := 0x08.
        methods := Method instances select: queryFilter.
    ];
    case: #tag do:
    [
        mode := 0x02.
        methods := Method instances select: queryFilter.
    ];
    case: #const do:
    [
        mode := 0x04.
        methods := Method instances select: queryFilter.
    ];
    case: #selector do:
    [
        mode := 0x01.
        methods := Method instances select: queryFilter.
    ];
    case: #selectorRefs do:
    [
        mode := 0x11.
        methods := Method instances select: queryFilter.
    ];
    case: #libVarRefs do:
    [
        mode := 0x20.
        methods := Method instances select: queryFilter.
    ];
    case: #browseCategoryRefs do:
    [
        mode := 0x40.
        methods := Method instances select: queryFilter.
    ];
    on: op.

   "See if we found any matches"
    methods size ifZero: [
       ^Notifier notify: 'No methods matched your query.'].

   "Put up the chooser window"
    (self@'input-link') linkInputProvider:
        (MethodChooser
            openOn: methods
            caption: caption ? ['Search results for: ',query]
            search: pattern).

>
> The latter is naturally much slower but I've implemented it as a
> background task, so when the results are in, a browser pops up, and I can
> have multiple searches going on simultaneously.
>
> Panu Viljamaa
>

-- Dave Simmons [www.qks.com / www.smallscript.com]
  "Effectively solving a problem begins with how you express it."

Panu wrote:
> Seems like Dolphin has a lot of momentum behind it.
> It might soon be the best Windows development platform, or is it already ?

Nearly...If I can just suss the MVP stuff, I'll probably say yes!