This quest is part of the Wyrm Hunt quests Ser Maximillian gives you. Select the option to "Rout a monster infestation" to start this quest.
Speak with the captain
Open the gate
Rout the goblins
Defeat the goblin chieftain
Report your victory.
The way to the Shadow Fort is a long and treacherous one. However, there's a shortcut that you can take to make the trip easier. You have to complete the sidequest "Of Merchants and Monsters" in the Ancient Quarry, due southwest of Gransys. Doing so will grant you safe access inside the Quarry, as well a merchant and replenishing treasures and gold pouches/bags.
After you exit, there’s also another camp to the southwest of the quarry’s exit where you can rest and manage your inventory. There’s also a dragon lurking to the southwest Shadow Fort. You may want to stick to the eastern path where you only have to deal with goblins and saurians.
Once you’re in the fort, talk to the captain and enter the hole on the ground. Follow it until you reach the exit and find yourself in the interior of a building inside the fort. Kill the goblins inside and collect the loot inside the room. Don’t forget to destroy the crates to get more items.
Head to the top floor make your way to the top floor and clear it. Take control of the ballistas there to take out the goblin-controlled ballistas in the main castle ahead. Be careful since the ballistas you’re controlling can also take damage and destroyed by those pesky goblins. If you managed to destroy the enemy-controlled ballistas, you can now use the ballistas to take out the cyclops below. Either that or make a run to the Station Room in the middle-west side of the fort.
Enter the station room and clear the prison cells. You’ll find the lever inside a chest there. Return to the previous building and attach the lever to the gate mechanism. Operate it to open the gate and let your allies inside.
Help them kill all the goblins in the area until the chieftain appears. Proceed inside the fort and take out the enemies inside. There’s a cyclops in the second floor balcony so be prepared to take it down. Find the chieftain hiding well inside the castle and slay it. Report back to Maximillian to complete the quest.