So you understand for the future, by default a Mac has three Library folders. These two are usually accessed by beginning at the root of your hard drive:
Generally, the first one is reserved for Mac OS X and third-party software won't put anything in there. Similarly, users shouldn't touch files in here unless they know what they're doing because an improper decision could lead to a non-functioning computer. The second one (/Library) is where software will install items to be used by all users. Again, this is an area users generally shouldn't go. Finally, there is the Library folder in your User folder. This will usually be listed in one of two ways:
Macintosh HD/Users/[Your User Name]/Library
The ~ symbol is a common shorthand for saying "within your user folder." Items in a user's Library folder apply only to that user. Removing things here typically leads to user settings being removed (e.g., application preferences, Address Book entries, calendar events, etc.).