Laravel按相关表排序

I have a little complicated relationship and I also want to order my results using column in the other table, let's see:

$book = Book::where('id', '=', $id)->with([
        'versions.titles' => function ($query) {

            //I want to paginate table "titles" 
            $query->groupBy('titles.title')->paginate(15);
        },
        'versions.titles.numbers' => function ($query) {

            // but I want to order it by "numbers.order_key"
            $query->orderBy('numbers.order_key', 'asc')->get();
        }])
        ->first();

Of course, this code doesn't work properly - titles column isn't sorted by numbers.order_key. Does anybody have any idea how to get it? I don't really want to use joins because of a complication of my relationships.

Update:

My models

Book:
-id

Title:
-id

Version:
-id
-book_id

titles_version:
-title_id
-version_id

Number:
-id
-book_id
-title_id

I want to obtain one nested record to paginate titles, e.g. $book->versions->titles->paginate(15), where titles have to be ordered by numbers.order_key.