In "The Devil and Tom Walker", what details about the swamp make it seem melancholy?

"Melancholy" is an interesting and appropriate word to use for the description of a location that has relevance to the mood of the writing. This term is often loosely defined as sad, gloomy or pensive, but distinguishes itself from these terms in that it was originally a medical diagnosis for long-term bad moods or depression. To describe a place as melancholy would thus imply that this isn't a subjective or periodic impression, but inherent to the nature of the place itself. 

On the first page, the swamp is also called a "morass". This is a term for a boggy, muddy place, but is also a homonym for confusion and complexity. Already we might begin to visualize the swamp as not only wet and messy, but also overgrown and maze-like; an unhappy place to be.

The swamp gets a little more attention shortly after Tom is introduced;

The swamp was thickly grown with great gloomy pines and hemlocks, some of them ninety feet high; which made it dark at noonday, and a retreat for all the owls of the neighborhood. It was full of pits and quagmires, partly covered with weeds and mosses; where the green surface often betrayed the traveler into a gulf of black smothering mud; there were also dark and stagnant pools, the abodes of the tadpole, the bull-frog, and the water snake, and where trunks of pines and hemlocks lay half drowned, half rotting, looking like alligators, sleeping in the mire.

The swamp thus appears simultaneously sleepy and deadly, and the fact that it is dark even in the middle of the day makes it seem impervious to the outside world, as if it's a separate dimension in which only the most unpleasant face of nature is shown. 

The swamp is next described as "treacherous" and, unsurprisingly, "melancholy", and to further reinforce the repulsive imagery, Irving injects details about black mold, rumors of human sacrifices and devil worship, and skeletal human remains, just prior to the appearance of Old Scratch, who is, of course, literally Satan himself. At this point he's really driving the imagery and symbolism as hard as he can, as if to reinforce that you really can judge this place by its appearance.