# DhbVector: implementation and extensions

27 messages
12
Open this post in threaded view
|
Report Content as Inappropriate

## DhbVector: implementation and extensions

 Hi all,I've been looking at DhbVector implementation and couple of questions came up:DhbVector is a sublcass of Array. Why not FloatArray? I see PolyMath has a package for automatic differentiation. If AD is used, the user will need the array of DualNumber's or Tape's (not implemented yet) In this case FloatArray might get on the way.Even more, why we even need a separate class for a vector? DhbVector doesn't have any instance or class variable. Vector operations can be implemented as extensions of Array/FloatArray.How I would like to see it:Have FloatArray implementing all the necessary operations for Float's only. The idea is that this would be an optimised implementation. In Common Lisp I would implement operations as multi-methods with type hints inside specialisations for floating-point arrays. I don't know how to achieve this in Pharo and what kind of optimisation it can offer.Have some kind of NumberArray, whose members are known to be some kind of numbers. So, it will be able to deal with DualNumber's, for example. Thus, it's less optimised, but more universal.I am not sure if this is achievable and what kind of design should be chosen to implement this with the minimum code (once again, in Common Lisp I would probably resort to macros as the code is essentially the same and only type hints are changing).On the top of this, to efficiently implement some extra linear algebra methods more BLAS-like operations on vectors are needed, i.e. operations that don't allocate the memory but modify existing vector. This can be used in, for example, implementation of Conjugate Gradients (CG) linear solver for positively defined symmetric matrices, BiCGStab and GEMRES for general matrices; these methods are efficient for sparse systems and can solve linear equations where matrix part is expressed as a function. I'm keen to add these operations: I can start adding them to DhbVector and if we decide to change the whole vector design, they can be moved to an appropriate class.Cheers,AlexeyPS: scmutils, the extension of MIT-Scheme, to study Classical Mechanics developed by G. Sussman, A. Radul and others from MIT uses an interesting idea for vectors: there are two kinds of vectors, columns (vectors, with indices in superscripts) and rows (co-vectors, with indices in subscripts); so, scalar multiplication would be achieved by multiplying a row by a column; if a column is multiplied by a row, this will give outer product. Furthermore, element-wise operations are only allowed for the same kind of vectors. -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

 On Monday, April 4, 2016 at 4:10:45 PM UTC+2, Alexey Cherkaev wrote:Hi all,I've been looking at DhbVector implementation and couple of questions came up:DhbVector is a sublcass of Array. Why not FloatArray? I see PolyMath has a package for automatic differentiation. If AD is used, the user will need the array of DualNumber's or Tape's (not implemented yet) In this case FloatArray might get on the way.Even more, why we even need a separate class for a vector? DhbVector doesn't have any instance or class variable. Vector operations can be implemented as extensions of Array/FloatArray.How I would like to see it:Have FloatArray implementing all the necessary operations for Float's only. The idea is that this would be an optimised implementation. In Common Lisp I would implement operations as multi-methods with type hints inside specialisations for floating-point arrays. I don't know how to achieve this in Pharo and what kind of optimisation it can offer.Have some kind of NumberArray, whose members are known to be some kind of numbers. So, it will be able to deal with DualNumber's, for example. Thus, it's less optimised, but more universal.I am not sure if this is achievable and what kind of design should be chosen to implement this with the minimum code (once again, in Common Lisp I would probably resort to macros as the code is essentially the same and only type hints are changing).On the top of this, to efficiently implement some extra linear algebra methods more BLAS-like operations on vectors are needed, i.e. operations that don't allocate the memory but modify existing vector. This can be used in, for example, implementation of Conjugate Gradients (CG) linear solver for positively defined symmetric matrices, BiCGStab and GEMRES for general matrices; these methods are efficient for sparse systems and can solve linear equations where matrix part is expressed as a function. I'm keen to add these operations: I can start adding them to DhbVector and if we decide to change the whole vector design, they can be moved to an appropriate class.Cheers,AlexeyPS: scmutils, the extension of MIT-Scheme, to study Classical Mechanics developed by G. Sussman, A. Radul and others from MIT uses an interesting idea for vectors: there are two kinds of vectors, columns (vectors, with indices in superscripts) and rows (co-vectors, with indices in subscripts); so, scalar multiplication would be achieved by multiplying a row by a column; if a column is multiplied by a row, this will give outer product. Furthermore, element-wise operations are only allowed for the same kind of vectors. -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

 In reply to this post by Alexey Cherkaev Hi Alexey,calculations with Floatarrays are of course faster but the precision is much worse than than normal arrays, imo it would be not so good to loose so much precision.wernerOn Monday, April 4, 2016 at 4:10:45 PM UTC+2, Alexey Cherkaev wrote:Hi all,I've been looking at DhbVector implementation and couple of questions came up:DhbVector is a sublcass of Array. Why not FloatArray? I see PolyMath has a package for automatic differentiation. If AD is used, the user will need the array of DualNumber's or Tape's (not implemented yet) In this case FloatArray might get on the way.Even more, why we even need a separate class for a vector? DhbVector doesn't have any instance or class variable. Vector operations can be implemented as extensions of Array/FloatArray.How I would like to see it:Have FloatArray implementing all the necessary operations for Float's only. The idea is that this would be an optimised implementation. In Common Lisp I would implement operations as multi-methods with type hints inside specialisations for floating-point arrays. I don't know how to achieve this in Pharo and what kind of optimisation it can offer.Have some kind of NumberArray, whose members are known to be some kind of numbers. So, it will be able to deal with DualNumber's, for example. Thus, it's less optimised, but more universal.I am not sure if this is achievable and what kind of design should be chosen to implement this with the minimum code (once again, in Common Lisp I would probably resort to macros as the code is essentially the same and only type hints are changing).On the top of this, to efficiently implement some extra linear algebra methods more BLAS-like operations on vectors are needed, i.e. operations that don't allocate the memory but modify existing vector. This can be used in, for example, implementation of Conjugate Gradients (CG) linear solver for positively defined symmetric matrices, BiCGStab and GEMRES for general matrices; these methods are efficient for sparse systems and can solve linear equations where matrix part is expressed as a function. I'm keen to add these operations: I can start adding them to DhbVector and if we decide to change the whole vector design, they can be moved to an appropriate class.Cheers,AlexeyPS: scmutils, the extension of MIT-Scheme, to study Classical Mechanics developed by G. Sussman, A. Radul and others from MIT uses an interesting idea for vectors: there are two kinds of vectors, columns (vectors, with indices in superscripts) and rows (co-vectors, with indices in subscripts); so, scalar multiplication would be achieved by multiplying a row by a column; if a column is multiplied by a row, this will give outer product. Furthermore, element-wise operations are only allowed for the same kind of vectors. -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

 example:a:=Array with:Float pi. f:=a asFloatArray . a squared.                "#(9.869604401089358)"f squared. "a FloatArray(9.86960506439209)" -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

 In reply to this post by Alexey Cherkaev Hi,re #scaleBy: long time ago i made #* commutatively usable with any mixture of scalars, dhbvectors and dhbmatrices and replaced all uses of #scaleBy: in dhb by #*. i thought using #* is conceptually simpler and one could perhaps one day deprecate #scaleBy: or so.werner -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

 oops, of course i mean the mixture of scalars with dhbvectors and dhbmatrices are commutatively mixable.On Tuesday, April 5, 2016 at 10:57:55 AM UTC+2, werner kassens wrote:Hi,re #scaleBy: long time ago i made #* commutatively usable with any mixture of scalars, dhbvectors and dhbmatrices and replaced all uses of #scaleBy: in dhb by #*. i thought using #* is conceptually simpler and one could perhaps one day deprecate #scaleBy: or so.werner -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

 On Tuesday, April 5, 2016 at 11:52:47 AM UTC+2, Alexey Cherkaev wrote: #scaleBy: does not produce a new vector as a result, but modifies the receiver. Whereas, I assume, it is expected that 'aNumber * aVector' doesn't modify aVector and answers a newly allocated vector. #scaleBy: in that sense is a lower level operation. Which brings me to the next point:Hi Alexey, yes, that is why i did not silently deprecate #scaleBy: . in dhb  Didier often uses in place calcs instead of producing intermediary objects, that then gets thrown away, but he often put them in a bigger method where several of these object updates are done in one iteration. wouldnt this be a possibility instead of a method that just updates y < - y + a * x etc?werner -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

 BLAS certainly makes sense if you want that basic linear algebra operations on numerical data to better scale(in terms of operations/second especially for large matrices, because it's not really sure it applies to 3x3 dimensions)If we want to compete with, say a basic Matlab interpreter or Numpy or R or etc..., then it's mandatory from my POV.If we are using a BLAS backend, then an idea is to create proxies and don't perform any operation untill it better matches a native BLAS operation. This is implemented in C++ for example in BOOST ublas http://www.boost.org/doc/libs/1_60_0/libs/numeric/ublas/doc/May I remind that I have developped a rather rough and dumb interface to BLACK/LAPACK for Smalltalk: Smallapacksee https://github.com/nicolas-cellier-aka-nice/smallapack and https://github.com/nicolas-cellier-aka-nice/smallapack/wikiIts status on latest Pharo is unknown, the package depends on old Compiler, but if there's interest into it I can inquire.Smallapack is not integrated in PolyMath because maybe it's too huge, and because interaction with dhb matrices/vector would have to be thought out first.I see it as a concurrent implementation, so one could choose either dhb or blas/lapack based matrix/vector implementation depending on the needs, and we could maintain a kind of minimal common API between the two.NicolasAs a disclaimer: I am definitely not in a position to make this call, I guess Stephan, Serge and Didier would need to decide. I can comment as a user: my potential use of PolyMath would include solution of ODE of size ~100, with minimization problem solution on the top of it. So, on one hand I don't need huge scalability. On the other hand, this problem requires an efficient foundation and BLAS/LAPACK may provide it.PS. What is the current state of FFI in Pharo? I saw there was a talk on this topic at PharoDays. -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

 In reply to this post by Alexey Cherkaev On Tue, Apr 5, 2016 at 6:36 PM, Alexey Cherkaev <[hidden email]> wrote: >> BLAS certainly makes sense if you want that basic linear algebra >> operations on numerical data to better scale >> (in terms of operations/second especially for large matrices, because it's >> not really sure it applies to 3x3 dimensions) >> If we want to compete with, say a basic Matlab interpreter or Numpy or R >> or etc..., then it's mandatory from my POV. >> >> If we are using a BLAS backend, then an idea is to create proxies and >> don't perform any operation untill it better matches a native BLAS >> operation. >> This is implemented in C++ for example in BOOST ublas >> http://www.boost.org/doc/libs/1_60_0/libs/numeric/ublas/doc/>> >> May I remind that I have developped a rather rough and dumb interface to >> BLACK/LAPACK for Smalltalk: Smallapack >> see https://github.com/nicolas-cellier-aka-nice/smallapack and >> https://github.com/nicolas-cellier-aka-nice/smallapack/wiki>> Its status on latest Pharo is unknown, the package depends on old >> Compiler, but if there's interest into it I can inquire. >> >> Smallapack is not integrated in PolyMath because maybe it's too huge, and >> because interaction with dhb matrices/vector would have to be thought out >> first. >> I see it as a concurrent implementation, so one could choose either dhb or >> blas/lapack based matrix/vector implementation depending on the needs, and >> we could maintain a kind of minimal common API between the two. >> >> Nicolas > > > As a disclaimer: I am definitely not in a position to make this call, I > guess Stephan, Serge and Didier would need to decide. I can comment as a > user: my potential use of PolyMath would include solution of ODE of size > ~100, with minimization problem solution on the top of it. So, on one hand I > don't need huge scalability. On the other hand, this problem requires an > efficient foundation and BLAS/LAPACK may provide it. This will be really great if we can have problems with size that run on PolyMath. We are user-driven here, please propose code and scenarios to do in that direction :-) -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Every DSL ends up being Smalltalk http://www.doesnotunderstand.org/-- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

 In reply to this post by Alexey Cherkaev Hi Alexey,i have to admit i could not exactly identify your mentioned updates (1 to 4) in that algo, but i assume we are talking about x(i), p(i), r(i) ?  they have all the same size, correct? in that case one could eg try for optimizing purposes to do a 1to:size do[:position| x(i) at:position put:whatever.p(i)at:position bla.r(i)bla] or something like that. (and one would probably do the a(i)*p(i) beforehand and perhaps some other minor points, i had just a short glance at this algo. the matrix multiplication could also be done stepwise in this thing, or? ). is that what you have in mind, or do i misunderstand you?werner -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

 In reply to this post by Nicolas Cellier On Tuesday, April 5, 2016 at 5:44:07 PM UTC+2, Nicolas Cellier wrote:Smallapack is not integrated in PolyMath because maybe it's too huge, and because interaction with dhb matrices/vector would have to be thought out first.I see it as a concurrent implementation, so one could choose either dhb or blas/lapack based matrix/vector implementation depending on the needs, and we could maintain a kind of minimal common API between the two.Hi Nicolas, i always feared that smallapack is to complicated to use for me, hence i never tried it. a minimal common API would probably help me overcoming that fear. (and i would need some simple installation instructions)werner -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.
Open this post in threaded view
|
Report Content as Inappropriate

## Re: DhbVector: implementation and extensions

 In reply to this post by werner kassens-2 i just noticed one cant do it in one go, beta has to be calced after the first go and in a second one p(i).On Tuesday, April 5, 2016 at 7:38:33 PM UTC+2, werner kassens wrote:Hi Alexey,i have to admit i could not exactly identify your mentioned updates (1 to 4) in that algo, but i assume we are talking about x(i), p(i), r(i) ?  they have all the same size, correct? in that case one could eg try for optimizing purposes to do a 1to:size do[:position| x(i) at:position put:whatever.p(i)at:position bla.r(i)bla] or something like that. (and one would probably do the a(i)*p(i) beforehand and perhaps some other minor points, i had just a short glance at this algo. the matrix multiplication could also be done stepwise in this thing, or? ). is that what you have in mind, or do i misunderstand you?werner -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout.