Electronic noise not only limits the performance of magnetic devices in practical applications but also provides valuable physical insights into these devices. The first part of this thesis discusses how the low frequency noise in magnetic tunnel junctions and giant magnetoresistance devices can be used to understand the fundamental noise sources. Previously, the low frequency noise in these systems has been reported to have an enormously large magnitude when the magnetization switches. This was attributed to magnetic fluctuations. An alternative mechanism of a slow drift in the device resistance is discussed, and we show how it produces noise spectra that are similar to those in previous reports. We conclude that this resistance drift causes a measurement artifact and the low frequency magnetic noise is not present in the measured samples within measurement error. As a second part of the thesis, we discuss a pronounced voltage dependent conductance feature present at nonzero bias in some magnetic tunnel junctions. The presence of this feature depends upon the oxidation condition for creating the barrier, and this effect is found to be interfacial in nature. We describe how the electronic structures and density of states at the barrier interfaces could be responsible for this effect, and possibility of utilizing the conductance measurement to probe the interfacial states.