Release 1.0.0 #251

jbcaillau · 2024-07-26T09:59:46Z

See control-toolbox/CTDirect.jl#115 (comment)

jbcaillau · 2024-08-01T08:38:50Z

@ocots hadn't you written a global list of todo's for 1.0.0? can't find nothing more than scattered issues (which is fine, but would like some prioritisation)

ocots · 2024-08-01T09:19:37Z

@jbcaillau I can't find any list. Maybe we can do one. I watch Léon Marchand at 11:40 and then, I will start a list.

ocots · 2024-08-01T12:50:25Z

jbcaillau · 2024-08-01T17:46:04Z

manu thanks @ocots I've added 🚀 on the must-be-done-before-1.0.0 according to me. @PierreMartinon @gergaud feel free to update!

ps. hey, #194 was the list I had in mind 👍🏽

ocots · 2024-08-01T20:29:27Z

For what I will do I have put a ⚽️

ocots · 2024-08-03T16:02:17Z

I have added MINPACK in the indirect simple shooting tutorial and the Goddard problem. I have to check that there is no error since I have added a try...catch to switch from hybrj to hybrd when an error occurs since with my computer I cannot use hybrj but hybrd is working.

ocots · 2024-08-04T10:00:41Z

We should also add a tutorial about how to improve performance installing HSL, using the best linear solver, etc.

jbcaillau · 2024-08-04T16:21:40Z

I have added MINPACK in the indirect simple shooting tutorial and the Goddard problem. I have to check that there is no error since I have added a try...catch to switch from hybrj to hybrd when an error occurs since with my computer I cannot use hybrj but hybrd is working.

@ocots nice. if not already done, I guess we can remove the dependency towards NonlinearSolve 💩

ocots · 2024-08-04T16:27:32Z

No there will be both.

jbcaillau · 2024-08-04T16:32:31Z

No there will be both.

where do we (still) need NonlinearSolvePoo?

ocots · 2024-08-04T16:56:00Z

No there will be both.

where do we (still) need NonlinearSolvePoo?

Tutorials:

Indirect simple shooting
Goddard

jbcaillau · 2024-08-04T17:24:13Z

OK: thought you had gone back to MINPACK for these.

ocots · 2024-08-05T07:17:02Z

I think we should add:

a tutorial about how to optimize the code, cf. https://github.com/0Yassine0/COTS.jl.
a tutorial (in manual part) about the solvers, cf. Document algorithms #253.
a tutorial (in manual part) about the use of the Flow function.

Actually the two first points are indicated in the list above. I should add something about the Flow function.

jbcaillau · 2024-08-05T10:31:40Z

I think we should add:

a tutorial about how to optimize the code, cf. https://github.com/0Yassine0/COTS.jl.
what do you mean? user stuff like changing the linear solver (HSL...), using MKL...?

we are far from optimal right now, but that's a dev (not user) issue, including (but not limited to):

building a better (AD)NLP model Pass sparse structure of constraints to NLPModels CTDirect.jl#183 (comment)
Use inplace when building the NLP model CTDirect.jl#188

a tutorial (in manual part) about the solvers, cf. Document algorithms #253.

the user cannot do much about ordering

a tutorial (in manual part) about the use of the Flow function.

right!

Actually the two first points are indicated in the list above. I should add something about the Flow function.

jbcaillau · 2024-08-05T10:35:16Z

To me only two things left to go 1.0.0:

@PierreMartinon 🚀 [Adjoint (dynamics multipliers) parsing CTDirect.jl#184](https://github.com/control-

@ocots 🚀 Flow with variables (NonFixed ocp) CTFlows.jl#15

Gents... 🙂

ocots · 2024-08-05T11:10:06Z

I think we should add:

a tutorial about how to optimize the code, cf. https://github.com/0Yassine0/COTS.jl.
what do you mean? user stuff like changing the linear solver (HSL...), using MKL...?

Yes, we could do something like: https://docs.sciml.ai/NonlinearSolve/stable/tutorials/code_optimization/

Just simple things like you said: changing the linear solver, using MKL. Saying that it is possible to change the AD backend without any comparison. It is maybe more tuning than optimising that we should say. But I think we can profit from what @0Yassine0 did to explain some possibilities. It shows the versatility of our package.

we are far from optimal right now, but that's a dev (not user) issue, including (but not limited to):

building a better (AD)NLP model Pass sparse structure of constraints to NLPModels CTDirect.jl#183 (comment)

Use inplace when building the NLP model CTDirect.jl#188

I was not thinking of this.

a tutorial (in manual part) about the solvers, cf. Document algorithms #253.

the user cannot do much about ordering

I was simply thinking about the fact that the user can choose the method to solve its problem. For the moment, there is only one case: see

OptimalControl.jl/src/solve.jl

Line 9 in 2ae8993

algorithms = add(algorithms, (:direct, :adnlp, :ipopt))

But, later there will be more. I thought about a simple tutorial explaining how to choose the method with its description. We could profit to redirect to the documentation of the packages we use, like ADNLPModels.jl, NLPModelsIpopt.jl and CTDirect.jl.

a tutorial (in manual part) about the use of the Flow function.

right!

Actually the two first points are indicated in the list above. I should add something about the Flow function.

jbcaillau · 2024-08-05T11:53:39Z

I was simply thinking about the fact that the user can choose the method to solve its problem. For the moment, there is only one case: see

OptimalControl.jl/src/solve.jl

Line 9 in 2ae8993

algorithms = add(algorithms, (:direct, :adnlp, :ipopt))

But, later there will be more. I thought about a simple tutorial explaining how to choose the method with its description. We could profit to redirect to the documentation of the packages we use, like ADNLPModels.jl, NLPModelsIpopt.jl and CTDirect.jl.

@ocots @PierreMartinon we should immediately add MadNLP: #259 Works out of the box (check #260) and @frapac can help in case of need 🙂

PierreMartinon · 2024-08-05T15:17:14Z

Regarding MadNLP, I have one question: I don't know how to have the CTSolveExt extension loaded if either NLPModelsIpopt or MadNLP are used. From the Julia doc, a package extension will load if all its dependencies are present.

We could try to have one extension per solver, but this is not how we currently work: we have a single function with a Symbol argument for the solver, not several methods with multiple dispatch...

PierreMartinon · 2024-08-05T15:29:36Z

For the sparsity pattern, the current trapeze method makes it a bit more involved: we typically store the dynamics at the end of a time step before moving on to the next step, so that we don't compute it twice. This complicates the decomposition of the constraints as a simple loop over the time steps, and the resulting sparsity pattern.

Here: https://github.com/control-toolbox/CTDirect.jl/blob/a9ae483a57382d44dba0612af914d78275fd6d7a/src/problem.jl

    args_i = ArgsAtTimeStep(xu, docp, 0, v)
    args_ip1 = ArgsAtTimeStep(xu, docp, 1, v)

    index = 1 # counter for the constraints
    for i in 0:docp.dim_NLP_steps-1

        # state equation
        index = setStateEquation!(docp, c, index, (args_i, args_ip1))
        # path constraints 
        index = setPathConstraints!(docp, c, index, args_i, v)

        # smart update for next iteration
        if i < docp.dim_NLP_steps-1
            args_i = args_ip1
            args_ip1 = ArgsAtTimeStep(xu, docp, i+2, v)
        end
    end

Ideally we would only have args_i and no update step. Maybe we should implement the implicit midpoint method and work from there.

ocots · 2024-08-05T15:58:46Z

@PierreMartinon Est-ce clair ?

Regarding MadNLP, I have one question: I don't know how to have the CTSolveExt extension loaded if either NLPModelsIpopt or MadNLP are used. From the Julia doc, a package extension will load if all its dependencies are present.

We could try to have one extension per solver, but this is not how we currently work: we have a single function with a Symbol argument for the solver, not several methods with multiple dispatch...

La première fonction solve se trouve dans les sources tandis que celle sur les types concrets se trouvent dans 2 extensions séparées.

Bon je me suis planté pour l'appel avec IpoptSolver mais tu vois l'idée.

jbcaillau · 2024-08-05T21:12:47Z

We could try to have one extension per solver [...]

@PierreMartinon that is what I would favor, again to avoid load time issues (= just load the relevant extension when triggered by the chosen solver).

jbcaillau · 2024-08-05T21:14:22Z

For the sparsity pattern, the current trapeze method makes it a bit more involved: we typically store the dynamics at the end of a time step before moving on to the next step, so that we don't compute it twice. This complicates the decomposition of the constraints as a simple loop over the time steps, and the resulting sparsity pattern.

@PierreMartinon oh, right. spares half the computation. nice.

ocots · 2024-08-06T12:41:22Z

@PierreMartinon bien sûr le solve correspond à ton solve_docp.

Voui. La je coince sur ces ù*^$ de descriptions: je voudrais rajouter et tester le cas :madnlp au lieu de :ipopt, mais je ne trouve pas la bonne syntaxe -_-
J'ai essaye de completer available_methods et de passer :madnlp au solve, mais pas moyen. C'est une liste de couples, ou bien une liste tout court ?

algorithms=()
algorithms = add(algorithms, (:adnlp, :ipopt))
algorithms = add(algorithms, (:adnlp, :madnlp))

available_methods()
MethodError: Cannot `convert` an object of type 
  Tuple{Tuple{Symbol,Symbol},Tuple{Symbol, Symbol}} to an object of type 
  Tuple{Tuple{Vararg{Symbol}}}

Pas sur non plus de comprendre comment fonctionne getFullDescription avec ses priorites.

Finalement est-ce qu'on ne ferait pas mieux de revenir a de bons vieux kwargs avec des valeurs par defaut ?

Edit: bon j'ai juste enleve le type de available_methods() et ca semble marcher

PierreMartinon · 2024-08-06T16:41:53Z

First madnlp tests control-toolbox/CTDirect.jl#191

jbcaillau · 2024-08-06T17:23:17Z

To me only two things left to go 1.0.0:

@PierreMartinon 🚀 [Adjoint (dynamics multipliers) parsing CTDirect.jl#184](https://github.com/control-

@ocots 🚀 Flow with variables (NonFixed ocp) CTFlows.jl#15

Gents... 🙂

@PierreMartinon one step away from 1.0.0 🤞🏽

control-toolbox/CTDirect.jl#184

ocots · 2024-08-07T20:08:12Z

@PierreMartinon bien sûr le solve correspond à ton solve_docp.

Voui. La je coince sur ces ù*^$ de descriptions: je voudrais rajouter et tester le cas :madnlp au lieu de :ipopt, mais je ne trouve pas la bonne syntaxe -_- J'ai essaye de completer available_methods et de passer :madnlp au solve, mais pas moyen. C'est une liste de couples, ou bien une liste tout court ?
algorithms=()
algorithms = add(algorithms, (:adnlp, :ipopt))
algorithms = add(algorithms, (:adnlp, :madnlp))
available_methods()
MethodError: Cannot `convert` an object of type 
  Tuple{Tuple{Symbol,Symbol},Tuple{Symbol, Symbol}} to an object of type 
  Tuple{Tuple{Vararg{Symbol}}}
Pas sur non plus de comprendre comment fonctionne getFullDescription avec ses priorites.

Finalement est-ce qu'on ne ferait pas mieux de revenir a de bons vieux kwargs avec des valeurs par defaut ?

Edit: bon j'ai juste enleve le type de available_methods() et ca semble marcher

Désolé le bon type était : Tuple{Vararg{Tuple{Vararg{Symbol}}}}.

jbcaillau · 2024-08-07T22:04:09Z

@ocots @PierreMartinon thanks for the active cleaning / upgrading (solve and everything) 👍🏽. was beach / boat day on may side 🌞

PierreMartinon · 2024-08-09T14:29:35Z

The parsing for the constraints multipliers should be up in the next release. Untested though.

Before 1.0 I'd like to

add a second discretization scheme (implicit midpoint). Adding other schemes later should be easier.
then try to pass the sparsity structure and see what performance we get.

jbcaillau · 2024-08-09T14:51:08Z

The parsing for the constraints multipliers should be up in the next release. Untested though.

Before 1.0 I'd like to

add a second discretization scheme (implicit midpoint). Adding other schemes later should be easier.

Not a priority, I think. I am not expecting big changes from midpoint (dual to trapezoidal scheme with more or less the same properties). Having a higher order would be nice (see control-toolbox/CTDirect.jl#101 (comment)) but not top priority for most i use case.

then try to pass the sparsity structure and see what performance we get.
more important, yes: see Pass sparse structure of constraints to NLPModels CTDirect.jl#183 (comment) (ADNLPModels way, instead of a fully manual process through NLPModels - if I got correctly)

But even more important (see this other exchange with @amontoison, control-toolbox/CTDirect.jl#188 (comment)) Check control-toolbox/CTBase.jl#232

PierreMartinon · 2024-08-09T15:05:13Z

I know, but I can't really pass the sparse structure with the trapeze method (more blocks than a true single step method due to each dynamics acting on two steps and not just one, and also the saving of this dynamics for the next step complicates things).

jbcaillau · 2024-08-09T15:53:25Z

@PierreMartinon ah ok. but I think you can. independently on how you (efficiently) you compute the constraint, its value is sth like

$$x_{i+1} - x_i - (t_{i+1} - t_i) / 2 (f(x_i, u_i) + f(x_{i+1},u_{i+1}))$$

for which the derivative wrt. $x_i$'s and $u_i$'s is clear (and structured). Agree?

jbcaillau · 2024-08-12T10:34:22Z

@PierreMartinon available for a quick call to decide on publishing 1.0?

PierreMartinon · 2024-08-13T10:39:42Z

@PierreMartinon available for a quick call to decide on publishing 1.0?

Eh, more likely next week. Do we have a deadline for 1.0, conf or something ?

jbcaillau · 2024-08-13T17:04:31Z

@PierreMartinon available for a quick call to decide on publishing 1.0?

Eh, more likely next week. Do we have a deadline for 1.0, conf or something ?

@PierreMartinon no deadline, just not bad to proceed according to what was planned above. next Monday? 15:00?

PierreMartinon · 2024-08-19T13:11:32Z

@PierreMartinon available for a quick call to decide on publishing 1.0?

Eh, more likely next week. Do we have a deadline for 1.0, conf or something ?

@PierreMartinon no deadline, just not bad to proceed according to what was planned above. next Monday? 15:00?

@jbcaillau Bump

jbcaillau · 2024-08-19T14:31:49Z

@PierreMartinon meeting scheduled tonight 21:00
https://univ-cotedazur.zoom.us/my/caillau

To discuss:

jbcaillau · 2024-08-19T14:33:11Z

@PierreMartinon meeting scheduled tonight 21:00 https://univ-cotedazur.zoom.us/my/caillau

To discuss:

Release 1.0.0 #251

Preparing for inplace NLP CTBase.jl#232

ADNLPModel new shiny stuff CTDirect.jl#111 (reply in thread)

@ocots @gergaud please join if you're around 🤞🏾

PierreMartinon · 2024-08-20T10:07:30Z

Breakage seems to work: control-toolbox/CTDirect.jl#209 (comment)

PierreMartinon · 2024-08-21T13:06:51Z

Constraints / multipliers parsing seems to work: control-toolbox/CTDirect.jl#210

jbcaillau assigned ocots Jul 26, 2024

jbcaillau mentioned this issue Jul 26, 2024

Ocp, docp, nlp models... control-toolbox/CTDirect.jl#115

Closed

jbcaillau changed the title ~~Release 1.0~~ Release 1.0.0 Aug 1, 2024

jbcaillau mentioned this issue Aug 1, 2024

Add Percival #255

Merged

Release 1.0.0 #251

Release 1.0.0 #251

Comments

jbcaillau commented Jul 26, 2024 • edited Loading

jbcaillau commented Aug 1, 2024

ocots commented Aug 1, 2024 • edited Loading

ocots commented Aug 1, 2024 • edited Loading

jbcaillau commented Aug 1, 2024 • edited Loading

ocots commented Aug 1, 2024 • edited Loading

ocots commented Aug 3, 2024

ocots commented Aug 4, 2024

jbcaillau commented Aug 4, 2024

ocots commented Aug 4, 2024

jbcaillau commented Aug 4, 2024

ocots commented Aug 4, 2024 • edited Loading

jbcaillau commented Aug 4, 2024

ocots commented Aug 5, 2024

jbcaillau commented Aug 5, 2024 • edited Loading

jbcaillau commented Aug 5, 2024 • edited Loading

ocots commented Aug 5, 2024

jbcaillau commented Aug 5, 2024

PierreMartinon commented Aug 5, 2024 • edited Loading

PierreMartinon commented Aug 5, 2024 • edited Loading

ocots commented Aug 5, 2024 • edited Loading

jbcaillau commented Aug 5, 2024

jbcaillau commented Aug 5, 2024

ocots commented Aug 6, 2024 • edited by PierreMartinon Loading

PierreMartinon commented Aug 6, 2024

jbcaillau commented Aug 6, 2024 • edited Loading

ocots commented Aug 7, 2024

jbcaillau commented Aug 7, 2024

PierreMartinon commented Aug 9, 2024

jbcaillau commented Aug 9, 2024 • edited Loading

PierreMartinon commented Aug 9, 2024

jbcaillau commented Aug 9, 2024 • edited Loading

jbcaillau commented Aug 12, 2024

PierreMartinon commented Aug 13, 2024

jbcaillau commented Aug 13, 2024

PierreMartinon commented Aug 19, 2024

jbcaillau commented Aug 19, 2024

jbcaillau commented Aug 19, 2024

PierreMartinon commented Aug 20, 2024

PierreMartinon commented Aug 21, 2024

jbcaillau commented Jul 26, 2024 •

edited

Loading

ocots commented Aug 1, 2024 •

edited

Loading

ocots commented Aug 1, 2024 •

edited

Loading

jbcaillau commented Aug 1, 2024 •

edited

Loading

ocots commented Aug 1, 2024 •

edited

Loading

ocots commented Aug 4, 2024 •

edited

Loading

jbcaillau commented Aug 5, 2024 •

edited

Loading

jbcaillau commented Aug 5, 2024 •

edited

Loading

PierreMartinon commented Aug 5, 2024 •

edited

Loading

PierreMartinon commented Aug 5, 2024 •

edited

Loading

ocots commented Aug 5, 2024 •

edited

Loading

ocots commented Aug 6, 2024 •

edited by PierreMartinon

Loading

jbcaillau commented Aug 6, 2024 •

edited

Loading

jbcaillau commented Aug 9, 2024 •

edited

Loading

jbcaillau commented Aug 9, 2024 •

edited

Loading