Good question. Sorcery went across the seas with Gandalf and the other Maiar, to the Undying Lands. However, I've basically assumed that arcane magic isn't all gone, by making a couple of assumptions, and/or minor tweaks to how the LotR ended.
Half of the magic of bards is arcane, but is the province of elvish blood, the first race to inhabit Middle Earth and learn directly from the Valar. Arcane magic from bards nowhere approaches the power of the Eldar elves who remained in the Undying Lands, nor the Noldor elves that returned to Middle Earth to war against Morgoth after he stole the Silmarils.
In the modifications I made, the Blue Wizards (two of Gandalf's Maiar peers) that traveled into the deserts of Harad taught the Haradrim about psionics, permanently altering the people that used it, but is very similar to arcane magic, to the point that Dispel Magic is the same as Dispel Psionics and each will affect the other type of power. So a psion can dispel a bard's spells, and vice versa where a bard can dispel a psion's powers.
The Easterlings have some shamanistic magic, some of which is arcane in nature (although different in scope than Gandalf), and so do the Shugenjas of the Easterling ruling clans. They were reputed to have their own "devilry", so that's not out of bounds.
Dwarves, who possess a type of arcane magic involving runes taught to them by Aule, their Valar creator, can craft non-aligned magic items. Dwarves had their own crafts in canon Tolkien, so it's not really much of a stretch here, either.
Basically, I'm straddling the uncomfortable position of being respectful of Tolkien's world (else I might as well call it Faerun instead of Middle Earth), while taking some side-avenues to avoid having to put characters into straight-jackets. But the two base classes that will most destroy the nature of Middle Earth and Tolkien's vision are none other than Wizards and Sorcerers, and there isn't much I can do about it. Once you get to 8th and 9th level spells, you are doing things that even Gandalf and Saruman had troubles accomplishing, and therefore lies the basic problem with actually using Middle Earth un-altered for D&D campaigns.
In contrast, because divine classes use spells and powers granted by the greatest of the divine beings, the Valar, it's easier to justify the awesome power of a Druid in Tolkien's world in the 4th Age since my modifications have the Valar set up as quasi-deities by the true deity Ao. Short of Ao himself, the Valar are the greatest beings in Tolkien's mythology, and are thus even greater than Gandalf and the other Maiar. So all things are possible with divine magic, but the arcane magic of the Maiar, specifically the arcane magic that allowed the creation of eldritch relics like the Silmarils and the Rings of Power left with their main practitioners, and what arcane magic is left is but an echo of the world-shaking power of Sauron, Balrogs, Gandalf, Saruman, and the Noldor elves such as Gil-Gilad, Feanor, Glorfindel, Galadriel and Elrond.
One of the design challenges I'm still working through is whether the psion class is also one of those Tolkien busters because a 3.5 psion can pretty much do everything a wizard can, at about the same levels. A compromise that might work would be to use the Psychic Warrior class to replace the psion since they can't teleport everywhere and travel planes of existence, and have powers that would greatly fit similar levels of a 3.0 Bard. A Shugenja might not work in this world, either, but I'm unlikely to let someone play a Shugenja until such design challenges are fully thought out and tested.
I realize this is all a bit wordy, but it's a lot more boiled down and condensed than reading the Silmarillion, the Lays of Beleriand, Tolkien's notes, and any other source I cobbled this together with...and you did ask.
But that does mean that some magic items created with arcane magic are craftable in this world, if done by classes like the bard (elves), psion (Haradrim) or other hybrid magic classes that have the requisite magic spells/powers required to craft them (the rules of the DMG on crafting magic items applies in intent, if not totally in exact practice). And there are aligned items that a divine spell-caster could make, or the non-aligned runed items created by Dwarves. So while two basic classes are excluded, there is still quite of lot of magic to explore as a Player Character. Magic, to Tolkien, appears to have been very subtle in the books, but very encompassing in the world and mythology of Middle Earth. He just didn't talk about it much in the stories covered by Peter Jackson's films, and it was mostly done by the divine beings, or those first races that learned directly from them. But that's not really what my 4th Age of Middle Earth is all about. Might as well do an epic campaign otherwise. My campaigns are for the rest of us, the non-divine beings that inherited Middle Earth after the War of the Rings ended.
"Are you sure you want to do that?" - Most important question I can ask you as a DM. So pay attention!
http://www.opengamingfoundation.org/srd.html (3.0 SRD)
http://www.d20srd.org/ (3.5 SRD)