Skipper Nicholas Pooran won the toss and opted to bat against a Zimbabwe side without captain Craig Ervine, who suffered a "mild asthmatic attack" before the game.
They were aggressive early on, but paid the price with Kyle Mayers (13) top-edging Blessing Muzarabani to wicketkeeper Regis Chakabva, who was deputising as skipper.
Johnson Charles found his groove and unleashed in the ninth over, hitting 6-4-4 off Ryan Burl to get the scoreboard moving before Evin Lewis mistimed a heave down the ground off Sikandar Raza and was caught.
Pooran fell for seven and when Charles was comically run out for 45 in a mix-up with Rovman Powell, the West Indies were reeling.
Zimbabwe kept the pressure on. Raza trapped Shamarah Brooks lbw, and then caught and bowled Holder with the West Indies losing four wickets for 11 runs in 12 balls to leave them on 101-6.
Powell (28) and Akeal Hosein (23 not out) steadied the ship and threw the bat in a late flourish to ensure they set a competitive target.
In reply, Zimbabwe slammed 18 in a nightmare first over from Mayers.
Undeterred, the Caribbean side made a breakthrough in the third over when Regis Chakabva dragged the ball onto his stumps off Joseph.
Joseph took his second with a lightning delivery to remove Tony Munyonga and Sean Williams fell two balls later as the pressure began building for Zimbabwe.
Allrounder Raza slammed 82 off 48 balls in Zimbabwe's first match on Monday, but he was out for 14 and when Joseph came back in the attack he shattered the stumps of Richard Ngarava and Luke Jongwe to end any hope they had.