Lynna pretty much nailed it. The context of James 4:4, if you just want to read one passage, is best read from 4:1-12.
James 4 (HCSB) wrote:1 What is the source of wars and fights among you? Don't they come from the cravings that are at war within you? 2 You desire and do not have. You murder and covet and cannot obtain. You fight and war. You do not have because you do not ask. 3 You ask and don't receive because you ask with wrong motives, so that you may spend it on your evil desires.
4 Adulteresses! Don't you know that friendship with the world is hostility toward God? So whoever wants to be the world's friend becomes God's enemy. 5 Or do you think it's without reason the Scripture says that the Spirit who lives in us yearns jealously?
6 But He gives greater grace. Therefore he says:
[indent]God resists the proud,
but gives grace to the humble.[/indent]
7 Therefore, submit to God. But resist the Devil, and he will flee from you. 8 Draw near to God, and He will draw near to you. Cleanse your hands, sinners, and purify your hearts, double-minded people! 9 Be miserable and mourn and weep. Your laughter must change to mourning and your joy to sorrow. 10 Humble yourselves before the Lord, and He will exalt you.
11 Don't criticize one another, brothers. He who criticizes a brother or judges his brother criticizes the law and judges the law. But if you judge the law, you are not a doer of the law but a judge. 12 There is one lawgiver and judge who is able to save and to destroy. But who are you to judge your neighbor?
The Holman Christian Standard Bible (my personally preferred translation) has a note on all the verses here, I'll post the notes on verses 4-5:
HCSB Study Notes wrote:4:4-5 Adulteresses refers to the congregations's unfaithfulness to God. Self-centeredness is cast as diametric opposition (hostility) to God. The phrase the Spirit who lives in us yearns jealously plays off "the cravings . . . at war within you" in verse 1, demonstrating the difference in attitude between a Spirit-filled life and one enslaved by selfish cravings.
Edit: apparently Atria sniped me, while being sniped by Lynna. Wow.