Hi @SteelWagstaff (answering for @rokmeglic). First I installed with composer. Now I installed dev (https://github.com/pressbooks/pressbooks-lti-provider/archive/dev.zip) version trough unzipping dev.zip in wp-plugins directory. Login in Chrome and Edge is still not working.
I have the latest Wordpress version, and Pressbooks LTI Provider 1.3.4 from dev branch (LTI deep linking is now working - thanks for resolving this issue). Pressbooks is version 5.17.2.