Wearable displays have advanced rapidly over the past few decades but they are limited in field-of-view due to optical constraints. Likewise, 3D displays have several technological and viewing discomfort limitations. These limitations result from the missing 3D depth cues in stereoscopic displays, which are essential for real 3D and for interactive augmented reality (AR) applications. Wear3D proposal aims to overcome the two fundamental scientific challenges of wearable displays and make them as natural as wearing a pair of eyeglasses: (i) Eliminate the relay lenses. We need to overcome the focusing problem of the eyes in order to completely eliminate the large relay lenses. As a result, miniaturization of wearable displays will be possible by taking full advantage of the advancements in micro-technologies; (ii) Provide all the essential 3D depth cues to avoid perceptual errors and viewing discomfort. We need to enable the two eyes to fixate at the correct depth of the objects rather than the display panel without losing resolution. Thereby, eliminating the conflict between the accommodation and convergence. Overcoming these challenges would enable a display which can provide natural looking and interactive 3D and very wide field-of-view (>100deg) in an eyeglasses form factor. Such a display goes far beyond the state-of-the art in wearable displays and open new research directions for intelligent human-computer interfaces and AR.