«Under review as a conference paper at ICLR 2016 CAPTURING MEANING IN PRODUCT REVIEWS WITH CHARACTER-LEVEL GENERATIVE TEXT MODELS Zachary C. Lipton ...»
In Advances in Neural Information Processing Systems, pages 3104–3112, 2014.
Under review as a conference paper at ICLR 2016 Tijmen Tieleman and Geoffrey E. Hinton. Lecture 6.5- RMSprop: Divide the gradient by a running average of its recent magnitude. https://www.youtube.com/watch?v= LGA-gRkLEsI, 2012.
Subhashini Venugopalan, Huijuan Xu, Jeff Donahue, Marcus Rohrbach, Raymond Mooney, and Kate Saenko. Translating videos to natural language using deep recurrent neural networks. arXiv preprint arXiv:1412.4729, 2014.
Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: A neural image caption generator. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3156–3164, 2015.
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In Advances in Neural Information Processing Systems, pages 3320–3328, 2014.
Xiang Zhang and Yann LeCun. Text understanding from scratch. arXiv preprint arXiv:1502.01710, 2015.
Under review as a conference paper at ICLR 2016 Appendices
A REVIEWS GENERATED BY ITEM CATEGORY
A.1 AMERICAN IPA
STRPours a transparent copper with a nice creamy off-white head that retains well and leaves behind a nice lace. S: Mild aroma of citrus and pine hops. The smell is not super pleasing. T: The body isn’t overly abused by the orange peel ﬂavors. It is a solid IPA that is followed by a mouthfeel with a huge rich malt background that is quite prevelent. The ﬁnish is slightly bitter on the tongue, and the malt is slightly dry by the end. A touch of sweet malts in the background.
M: Great carbonation and a light moderate carbonation feel. Flavors are very pleasant nicely balanced by the hops and resins. D: This is an easy drinking IPA and I would deﬁnitely like to try it again for what it is anyway. Overall: This is a well crafted beer. It is a great example of an IPA, I would deﬁnitely have another one of these and I will be getting another can in the bottle and like to see it more in the can in some time.EOS
A.2 AMERICAN ADJUNCT LAGER
STRA - Poured a dark yellow color with a white head that leaves lacing behind S - Lightest aroma, some corn with a slight hop presence. It has a fairly strong aroma of corn and yeast. The taste is slightly sweet and crisp, corn and a light hop presence. The ﬁnish is moderately bitter with a slight sweetness. Finishes sweet and clean with a light crisp ﬁnish. Not much more to say. This would be a good one to have when ice cold, and I’d rather have a Coors Light.EOS
A.3 RUSSIAN IMPERIAL STOUT
STRThanks to Buttlepoptu for the chance to try this one. Got this bottle from a 12 oz. bottle in a tasting glass. A- Pours a deep black color with a miniscule head that dissipates quickly. No lacing to speak of either. S- Nice bitter chocolate nose. Lots of chocolate, coffee, roasted coffee, and a little chocolate. T- The dark roasted malts provide a sweet ﬁnish, and in the most refreshing taste and the bitterness of this one is a little subdued. M: Thick and rich, like syrup. Slick and creamy. Sits right away and it lasts for just a little D: What a world class RIS.
While I love all the world of beers that I’ve tried in the past, this is very drinkable.
I think it could be a little much for sure. Still a good beer for sure. This one is worth the wait. While it is not a beer I could drink a few of without getting a little difﬁcult to drink and ﬁnish the whole bottle. EOS
A.4 AMERICAN PORTER
STR Served a beautiful black with a mocha head that started about 1/1 inch thick and left lots of lacing. The smell was sweet and roasty with some vanilla and a hint of chocolate. The taste was primarily that of smoky chocolate and some chocolate and barrel character. It had a light mouthfeel with a crisp carbonation.
Overall this was a very good beer. I was more interested in pouring this one from a bottle though it is a very good brew (that a must have again). I have another one on tap and it was my ease of time. Overall: A big thanks to Richmond for the chance to try this one. I’ve enjoyed other porters to be but this is deﬁnitely a porter. EOS Under review as a conference paper at ICLR 2016 A.5 FRUIT/VEGETABLE BEER STROn tap at the brewpub. A nice dark red color with a nice head that left a lot of lace on the glass. Aroma is of raspberries and chocolate. Not much depth to speak of despite consisting of raspberries. The bourbon is pretty subtle as well.
I really don’t know that I ﬁnd a ﬂavor this beer tastes like. I would prefer a little more carbonization to come through. It’s pretty drinkable, but I wouldn’t mind if this beer was available. EOS Under review as a conference paper at ICLR 2016
B REVIEWS GENERATED BY STAR RATINGBelow, we show examples of the concatenated input network’s ability to generate decidedly positive reviews when conditioned on high star ratings and decidedly negative reviews when conditioned on low ratings. Because the star rating is simply a scalar input to the network, we can also generate reviews conditioned on the more extreme 0 star and 10 star ratings, even though neither truly exist in the dataset. We present examples of reviews generated by the model and conditioned upon 0, 1, 5, and 10 star ratings. For each we show an example each for three settings of the temperature (.1,.5, 1.0).
B.1 0 STAR REVIEWS The following full reviews are generated conditioned on a 0 star rating for varying settings of the temperature.
With a temperature of.1 STRPours a clear amber with a thin white head that dissipates quickly. Smell is of caramel malts and some hops. Taste is a little sweet and malty. The mouthfeel is thin and watery. This is a very disappointing beer. I would not buy this again.EOS
With a temperature of.5:
STRThis is an old bottle of a premium beer. This was a bad batch. Pours a color that is black. The head is a small bubbling of bubbles with a thick ring and leaves some nice lacing. The aroma is a strong metallic twang and some light chocolate notes. The smell is of cocoa and coffee. While the aroma is an astringent blast of chocolate and coffee on the nose. The taste is also ﬁlled with a very strong alcohol ﬂavor. It tastes like a bad barleywine. The ﬂavors are very strong, and the malt and dark fruits take over the ﬂavor. The mouthfeel is thin and watery. The drinkability is not good.EOS
With a temperature of 1.0
STRThe color is a rice golden the body with ﬁzzy yellow carbonation (which is unpleasant). No lacing, which suffers from half an inch. Anyway: horrible, suck, surprise, and unpleasant. I below a dubbel, but the honey seems a bit distracting. Thin almost taste of baked bread, caramel malt and almonds...very sickly without much addition into the bottom of perhaps in my mouth. This beer conversely, but then again my rear sourness really kicked up the malt and lies. Mouthfeel was ﬁne. I was difﬁcult to ﬁnish this ﬁxe a year and thank Guinness at a party beer. I say this is one very watered down beer.EOS B.2 1 STAR REVIEWS With a temperature of 0.1 STRPours a clear amber with a thin white head. Smell is of sweet malt, and a hint of caramel. Taste is also sweet and malty with a slight hop bitterness and a slight bitterness. Mouthfeel is medium bodied with a smooth and creamy mouthfeel. This is a beer that I will probably not go out of my way for any time soon.EOS With a temperature of 0.5
STRA: Pale yellow color with a small white head. S: Spicy and citrusy. T:
Sweet and sour fruit with a bit of a metallic taste at the end. M: Light, sticky, and syrupy. D: Somewhat tasty, but the alcohol does not have a lot of complexity to the beer. I would not buy this again.EOS Under review as a conference paper at ICLR 2016 With a temperature of 1.0 STRThis beer was gone alright. It did not go away. It’s all about the booze, but it seems a rather peppery, aggressive, heavy, smelling (at times) for something better. No where near as big as I’d like. Artiﬁcially. You would love this beer...
even a sipper. If kolsh were an oddball sourness, I’d swear this one would win. I haven’t been afraid of my fridge selection for less money.EOS B.3 5 STAR REVIEWS With a temperature of 0.1 STRPoured from a 12 oz. bottle into a pint glass. Appearance: Pours a clear golden color with a thick frothy white head that slowly fades to lacing. The aroma is great. It has a sweet caramel malt aroma with a slight hop aroma. The taste is a little sweet with a slight hop bitterness that lingers in the aftertaste. The mouthfeel is medium bodied with a good amount of carbonation. This is a very drinkable beer. I would have no problem drinking a few of these in a sitting. I would deﬁnitely drink this again in the future.EOS
With a temperature of 0.5
STRA - Poured a midnight black with a thick frothy tan head that left lacing on the glass. S- Smell is a wonderful virtual malt aroma with a touch of coffee and chocolate. Taste is chocolate and coffee with a hint of chocolate and coffee.
Its a little thin for a witbier at all. M- This beer has a medium mouthfeel with a slight amount of carbonation. D - Very drinkable beer. I could drink this all night long. I will deﬁnitely be buying a few more in the cellar.EOS
With a temperature of 1.0
STRFirst off, I have batch whose pure vertical esopetime trade! The forgot ”Homaching copper” IPA would be a popular beer ﬁxed in Pounds VE! Poured into Duvel tulip. APV of caramel with a small light tan head that cakes off on the glass now and lays down to a thin layer of residence that leaves behind dots of lace as it settles. Several left behind with remarkable introducing curtains. Aroma is enveloping massive but incredibly were redolent – refreshing. Juicy Belgian yeast esters - bursts of lime, lemon pith, etc... all right up front. Spiciness is in a glorious essence. Really man’s Belgian pale and wild yeast funk too. Basically the taste’s aroma is mostly grapefruit juice and vegetably accents. hints of oak, cranberries and light hints of yeast funk. A faint funkiness in there as well, with all malts. And smooth and a bit muddy too. Finish is fresh and both easy to glass, you know the glass who dies on that sweetness itself, but you still think you what effect you get - it makes the least busy-love color? Stone has perfect! Mouthfeel is smooth and clean with an earthy smoothness that lack the nice sharp bite of the malt and certain to exquisitely surface from the malt. Exceptional imperial stout. Low to drinkability, in an awesome winter warmer, especially when a bit of age doesn’t work for me. A 20% beer is only great by my someone worry but a beer that would go well with something along with my friend. Nicely done in any mark. Superb.EOS B.4 10 STAR REVIEWS With a temperature of 0.1 STRPoured from a 12 oz. bottle into a pint glass. Appearance: Pours a dark brown color with a thick tan head that lasts forever and leaves a nice lacing on the glass. The aroma is of chocolate, coffee, chocolate, and a hint of chocolate. The taste is a nice blend of coffee, chocolate, and coffee. The mouthfeel is smooth and creamy with a nice creamy feel to it. This is a very good beer that I would
Under review as a conference paper at ICLR 2016
deﬁnitely have again. I would love to have this again and again. I wish I had more of this stuff and see how this is the best beer I’ve ever had. I could drink this all day and enjoy it all the time. I could drink this all day and drink all day. I could drink this all day and drink all day and savor the other three of these and this is a staple in my fridge.EOS
With a temperature of 0.5
STRA- Amber brown with a small white head that has decent retention and leaves a nice lacing on the glass. Aroma is strong of citrus hops and a mild sweet malt aroma. The taste is so smooth and crisp tasting with the hops throughout the taste. The malt is more citrus than I expect to be sure this is a hop bomb with the alcohol in this beer that makes it easy to drink and smooth. The alcohol is noticeable and delicious. This is a great beer, and a great take on the style and one of the best in the world. I could drink this all night. I guess the alcohol is evident in the ﬂavor and the ﬂavors are bright and crisp and drinks smoother and is so easy to drink. I wish I had more! This is the best beer that I’ve ever had in awhile. I would deﬁnitely order it again and again.EOS
With a temperature of 1.0
STRA: Pours vibrant amber with a nice, hugs foamy head. Awesome light brown sheen that looks like it. successful excellent! Want down thoroughly IPA indeed. Soft with pine and lime juice (since you sat down!) come through at most.
Also kicking bach down from something dampeness. Take through the enjoyment of this is wonderful for sure. Delicious. Is this rank? Now that it full body’s imperessively. Espresso-strock, this beer is that fantastic. You will nip sip that OK. I drink one beer yet! I can’t remember the 2008 version I might even paybe local!EOS Under review as a conference paper at ICLR 2016
C CATEGORY TRACKING OVER SENTENCESHere we show several representative examples demonstrating the ability of the concatenated input network (run in reverse) to recognize the category of the beer at the precise point in the sentence when it becomes clear. At ﬁrst the probabilities are all close to.2 reﬂecting the uniform prior. By the end of the sentences the distribution conditioned on the input is considerably less entropic.
Figure 5: We plot the probabilities of the 5 beer categories after each character in the sentence is encountered.
Under review as a conference paper at ICLR 2016