FREE ELECTRONIC LIBRARY - Online materials, documents

Pages:     | 1 |   ...   | 2 | 3 ||


-- [ Page 4 ] --
As each character in a review is encountered, we can plot the rating (with a granularity of 100 evenly spaced settings between 1 star and 5) which gives the review highest likelihood. Thus we can tell not only the sentiment of the rating, but the precise word, and even character at which this sentiment became clear.

–  –  –

Figure 6: We plot the argmax of the review’s likelihood over many settings of the rating.

Under review as a conference paper at ICLR 2016


–  –  –

Figure 7: We plot the likelihood given to a review by each rating. The network learns nonlinear dynamics of negation. “Not” reduces the rating when applied to “great” but increases the rating when applied to “bad”.

Under review as a conference paper at ICLR 2016


We trained a concatenated input RNN, with item category information as the auxiliary input. Inferring the class probability via the conditional likelihoods of the review, we can use the model in reverse to predict the category of the beer described in the review. Using a balanced test set of 5000 reviews, we evaluated the classification performance of the category RNN against two multinomial regression classifiers, one trained on the top 10,000 n-grams from the training set, and the other trained on tf-idf transfromed n-grams. The confusion matrices for these experiments can be seen in Table 2, Table 3, and Table 4. We also show results for a concatenated input RNN with rating information as used to classify positive (≥ 4.0 stars) and negative (≤ 2.0 stars) reviews.

–  –  –

Table 7: n-gram tf-idf positive/negative classification results on balanced dataset.

Under review as a conference paper at ICLR 2016


In this paper, we introduced and demonstrated the efficacy of a simple technique for incorporating auxiliary information xaux in a generative RNN by concatenating it with with the character repret) sentation xchar at each sequence step. However, sometimes we don’t simply want to generate given a representation xaux, but to learn a representation of xaux. For example, to generate a characterlevel caption given an image, we might want to jointly learn a convolutional neural network to encode the image, and a generative RNN to output a caption.

To accomplish this task at the character level, we propose the following network architecture and hypothesize that is will provide the benefits of learning an encoding while preserving our ability to generate long passages at the character level. At train time the xaux is fed to an encoder, whose output is then passed as auxiliary information to a concatenated input network. At prediction time, for any input, the encoding is calculated once, after which the inference problem is identical to that of our demonstrated concatenated input network.

Figure 8: Generative model with input replication. We train the network to produce a 5 star review by concatenating the rating with the one-hot representation of each character.

Under review as a conference paper at ICLR 2016


For each task, we train two unsupervised character-level donor RNNs so that we may harvest the weights for transplantation into the concatenated input networks. We train separate donor networks for the two tasks (rating and category modeling) because each is trained on a different subset of the data (the beer set is selected for class balance among the 5 categories and is thus smaller). These networks are trained until convergence (Figure 9). After transplantation, we train the concatenated input RNNs with high learning rates, to induce the weights for auxiliary information to grow quickly.

This results in an initial spike in loss (to quick to be seen in Figure 10), after which the loss quickly decreases.

–  –  –

Figure 9: The learning curves for the unsupervised donor character RNNs used for weight transplantation.

(a) Rating network after transplantation (b) Category network after transplantation

Pages:     | 1 |   ...   | 2 | 3 ||

Similar works:

«Látható beszéd: beszédhang alapú fejmodell animáció siketeknek Feldhoffer Gergely1, Bárdi Tamás1 Pázmány Péter Katolikus Egyetem, Információs Technológiai Kar, 1083 Budapest, Práter utca 50/a, Magyarország {flugi, bardi}@itk.ppke.hu Kivonat: Elkészült egy fej-animációs rendszer, ami siketek számára beszédjelb l olyan szájmozgást állít el, hogy a siket felhasználó azt megérthesse. Egy olyan audiovizuális adatbázis készült el hozzá, amihez professzionális...»

«Martin Luther's Quiet Time Walter Trobisch wrote the following delightful, but brief volume, in 1975. We are most appreciative to his widow, Ingrid Trobisch, for giving us permission for this reprint. Martin Luther had a barber. His name was Peter Beskendorf. One day Master Peter must have taken the liberty of asking his world-famous customer and doctor of theology, Dr. Luther, how do you pray? And Martin Luther answered. It was not beneath him to write a long letter to his barber a letter of...»

«MASH I LE SOMMELIER I UMAMI I UMAMIGO WHAT IS COPENHAGEN CONCEPTS? Copenhagen Concepts ApS is the parent company behind the successful international restaurants MASH, Le Sommelier, Umami and UmamiGo. Copenhagen Concepts owns all the restaurants and the rights to the brands and serves as the administrative company overseeing the entire group. The group currently has 630 employees and earns a revenue of USD 60 million. JANUARY 2015 BRAND IDENTITY Of course, fundamental to any restaurant is the...»

«Town of Chester – Board of Selectmen’s Meeting – December 17th, 2009 Town of Chester Board of Selectmen Meeting Thursday, December 17th, 2009 Municipal Complex Approved Minutes I Preliminaries 1.1 Call to Order Chairman Hassam called this meeting of the Town of Chester Board of Selectmen to order at 7:03pm.1.2 Roll Call Selectmen present: Robert Brown Jack Cannon Joe Castricone James Hassam Stephen Landau Members of the Public present, at various times: Barbara Dolloff, Welfare Officer...»

«FEDERAL RESERVE SYSTEM Docket No. OP1445 DEPARTMENT OF THE TREASURY Financial Crimes Enforcement Network 31 CFR Part 1010 RIN 1506-AB20 Final Rule: Definitions of Transmittal of Funds and Funds Transfer AGENCY: Financial Crimes Enforcement Network (“FinCEN”), Department of the Treasury; Board of Governors of the Federal Reserve System (“Board”). ACTION: Final Rule. Final Rule: Definitions of Transmittal of Funds and Funds Transfer SUMMARY: The Financial Crimes Enforcement Network, a...»

«Towards Consolidation, Inclusion and Stability FINANCIAL STABILITY REPORT Nepal Rastra Bank Central Office Baluwatar, Kathmandu January 2014 FINANCIAL STABILITY REPORT (January 2014  Issue No. 4) Nepal Rastra Bank Central Office Baluwatar, Kathmandu Disclaimer This fourth issue of the Financial Stability Report is based on the provisional data of Bank & Financial Institutions (BFIs) and other financial institutions as of mid-Jan 2014. Data used in its analysis may thus differ from the most...»

<<  HOME   |    CONTACTS
2017 www.thesis.dislib.info - Online materials, documents

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.