This is probably the worst party / invite / group system I've ever seen on PS4 - coming from Destiny, which is very good - and then The Division which is an absolute masterpiece (in this aspect), I expected a lot more from Blizzard. Thankfully the rest of the game is pretty good
Inviting others is pretty easy as you've found. Go to social, X on them, Invite to group. You can also do it from press triangle on the main screen, "Edit" next to your player icon I think it says.
The invited player will then get a PS4 system message. It appears top left, and while it's there a press on the central PS button on the controller will take you to it where you can accept the invite. This is the easiest way. PS button again (or navigate dashboard) and when back in the game you will be in group.
If you miss the window, press PS button to go to the dashboard, go up to the top row and left along to notifications. From here you can find the invite message and accept as above. PS button again (or navigate dashboard) and when back in the game you will be in group.
When grouped, you can see icons for each player in the top right. One has a green crown, this is the group leader (the one who sent out the initial invites, usually). Group leader takes you in to games.
I haven't found how to leave games other than everyone leaving individually then going through the invite/group up routine again. Terrible lol.
This game does have chat audio - I have heard other players while solo on a very rare occasion. Due to the terrible grouping system you are MUCH better off making a "Party", again from the top row of the dash (similar invite system, but once a party is running you can see it in the list / just jump in if you're friends with at least 1 member). A chat party remains as people drop in/out of in-game group, so is very preferable for organisation. Sound quality is superior too, in general.