feat: allow to-many ordering

[Fridious]

This feature implements the possibility to use many-to-many relation ordering.

[steve-chavez]

Since there are multiple rows in the to-many end, how do we decide which row we order into?

[Fridious]

what do you mean exactly? can you please give me an example?

[laurenceisla]

I may be misinterpreting @steve-chavez, but in this example the nested relationship has two rows for the first client:

"projects":[
  {"id":1,"name":"Windows 7","client_id":1},
  {"id":2,"name":"Windows 10","client_id":1}
]

So, does it take {"id":1,"name":"Windows 7", ...} or {"id": 2,"name":"Winodws 10", ...} to order the top relationship?

From what I see in other example, it should order the top relationship according to the first element of the nested relationship (in this case {"id": 1 ...}}), right? Or does it behave in a different way?

[Fridious]

Now I understand what you mean. It works like the internal postgres array sorting mechanism. I attached an example image from my application. For your info, the application uses the uuid behind the displayName of the account to sort.

Example matrix:
data:
Name|Accounts
Person-2|Account-2, Account-1
Person-1|Account-1, Account-2
Person-3|Account-2

results in:
Person-1|Account-1, Account-2
Person-2|Account-1, Account-2
Person-3|Account-2,

[steve-chavez]

From what I see in other example, it should order the top relationship according to the first element of the nested relationship (in this case {"id": 1 ...}}), right? Or does it behave in a different way?

@Fridious Could you answer/clarify the above? I don't quite get your example.

[laurenceisla]

OK, so basically this is the main query without the WITH pgrst wrapper that log-query = true shows:

SELECT "test"."clients".*,
       COALESCE("clients_projects_1"."clients_projects_1", '[]') AS "projects"
FROM "test"."clients"
LEFT JOIN LATERAL
 (SELECT json_agg("clients_projects_1" ORDER BY "clients_projects_1"."id" ASC)::jsonb AS "clients_projects_1",
         array_agg("clients_projects_1"."id" ORDER BY "clients_projects_1"."id") AS "clients_projects_1_ord_1"
  FROM
    (SELECT "projects_1".*
     FROM "test"."projects" AS "projects_1"
     WHERE "projects_1"."client_id" = "test"."clients"."id"
     ORDER BY "projects_1"."id" ASC) AS "clients_projects_1") AS "clients_projects_1" ON TRUE
ORDER BY "clients_projects_1"."clients_projects_1_ord_1"

So it would order by the result of the array_agg(...) aggregate (an integer[] in this case). However it's a bit confusing when using both orderings at the same time, should the first ordering affect the other? Like for example, add a new project here:

insert into test.projects values (20,'Test',1);

Ordering the internal by projects.order=id.desc and then the top by order=projects(id).asc, it shows the "id": 20 first instead of the ones with less id:

curl 'localhost:3000/clients?select=*,projects(*)&projects.order=id.desc&order=projects(id).asc'

[
  {"id":1,"name":"Microsoft","projects":[
    {"id": 20, "name": "Test", "client_id": 1},
    {"id": 2, "name": "Windows 10", "client_id": 1},
    {"id": 1, "name": "Windows 7", "client_id": 1}
  ]},
  {"id":2,"name":"Apple","projects":[
    {"id": 4, "name": "OSX", "client_id": 2},
    {"id": 3, "name": "IOS", "client_id": 2}
  ]}
]

Is this expected? If so it looks confusing from a user perspective.

[Fridious]

Maybe I explained it a little too confusing. To me, this looks correct.
If you change the query to &projects.order=id.desc&order=projects(id).desc, then the result should be the same because the nested objects are responsible for sorting the parent object. I didn't change any of the code for this. It's the same as before.

[laurenceisla]

OK, so order=projects(id).asc is the main order here and then the projects.order=id.desc is done after or separately from that. Makes sense from the perspective of: "ordering the nested resource does not affect the top order", however this could be confusing at first glance IMO... this would need a doc explanation, I think.

It's the same as before.

There are many new things here since we didn't allow to-many ordering before, that's why we're doing many questions here to understand better the implementation. The next step would be to check the integration with other features like aggregates, spread operator, nested resources, etc. I'll be reviewing those and check if anything breaks (these would need to be added to tests too).

[Fridious]

Got it, thanks for the clarification. Please let me know if there’s anything you’d like me to change or add on my side.

[laurenceisla]

For starters, this would need a Documentation entry, but in order to do that we need to define the use case with clarity as mentioned here #4592 (comment) (what exactly does the user expect to get with this feature).

[wolfgangwalther]

All these test case changes are changing something like order=x(y) to order.x=y. But that's not the same, right?

I believe the order.x=y ordering worked before already and just sorts the embedding, not the top-level.

[Fridious]

Maybe. I will adjust the tests. thanks

[Fridious]

for my personal use case, I can now use these queries:
curl "http://localhost:3000/person?select=sys_id,first_name,last_name,relation_owned_persons(source_id,reference:sys_user(sys_display_name))&order=relation_owned_persons(source_id).asc&offset=0&limit=10"

[laurenceisla]

for my personal use case

It would be nice to have the use case explained here for better understanding of the feature, this will be useful for the documentation too. I understand from your explanations that you're going with: "That's how SQL orders it", which is fair for us who are accustomed with the source code and know how the query is built under the hood. But for the end user there would need to be an explanation of the use case, IMO.