Source Code Cross Referenced for ArticleFilterPlugIn.java in  » RSS-RDF » curn » org » clapper » curn » plugins » Java Source Code / Java DocumentationJava Source Code and Java Documentation

Java Source Code / Java Documentation
1. 6.0 JDK Core
2. 6.0 JDK Modules
3. 6.0 JDK Modules com.sun
4. 6.0 JDK Modules com.sun.java
5. 6.0 JDK Modules sun
6. 6.0 JDK Platform
7. Ajax
8. Apache Harmony Java SE
9. Aspect oriented
10. Authentication Authorization
11. Blogger System
12. Build
13. Byte Code
14. Cache
15. Chart
16. Chat
17. Code Analyzer
18. Collaboration
19. Content Management System
20. Database Client
21. Database DBMS
22. Database JDBC Connection Pool
23. Database ORM
24. Development
25. EJB Server geronimo
26. EJB Server GlassFish
27. EJB Server JBoss 4.2.1
28. EJB Server resin 3.1.5
29. ERP CRM Financial
30. ESB
31. Forum
32. GIS
33. Graphic Library
34. Groupware
35. HTML Parser
36. IDE
37. IDE Eclipse
38. IDE Netbeans
39. Installer
40. Internationalization Localization
41. Inversion of Control
42. Issue Tracking
43. J2EE
44. JBoss
45. JMS
46. JMX
47. Library
48. Mail Clients
49. Net
50. Parser
51. PDF
52. Portal
53. Profiler
54. Project Management
55. Report
56. RSS RDF
57. Rule Engine
58. Science
59. Scripting
60. Search Engine
61. Security
62. Sevlet Container
63. Source Control
64. Swing Library
65. Template Engine
66. Test Coverage
67. Testing
68. UML
69. Web Crawler
70. Web Framework
71. Web Mail
72. Web Server
73. Web Services
74. Web Services apache cxf 2.0.1
75. Web Services AXIS2
76. Wiki Engine
77. Workflow Engines
78. XML
79. XML UI
Java
Java Tutorial
Java Open Source
Jar File Download
Java Articles
Java Products
Java by API
Photoshop Tutorials
Maya Tutorials
Flash Tutorials
3ds-Max Tutorials
Illustrator Tutorials
GIMP Tutorials
C# / C Sharp
C# / CSharp Tutorial
C# / CSharp Open Source
ASP.Net
ASP.NET Tutorial
JavaScript DHTML
JavaScript Tutorial
JavaScript Reference
HTML / CSS
HTML CSS Reference
C / ANSI-C
C Tutorial
C++
C++ Tutorial
Ruby
PHP
Python
Python Tutorial
Python Open Source
SQL Server / T-SQL
SQL Server / T-SQL Tutorial
Oracle PL / SQL
Oracle PL/SQL Tutorial
PostgreSQL
SQL / MySQL
MySQL Tutorial
VB.Net
VB.Net Tutorial
Flash / Flex / ActionScript
VBA / Excel / Access / Word
XML
XML Tutorial
Microsoft Office PowerPoint 2007 Tutorial
Microsoft Office Excel 2007 Tutorial
Microsoft Office Word 2007 Tutorial
Java Source Code / Java Documentation » RSS RDF » curn » org.clapper.curn.plugins 
Source Cross Referenced  Class Diagram Java Document (Java Doc) 


001:        /*---------------------------------------------------------------------------*\
002:          $Id: ArticleFilterPlugIn.java 7041 2007-09-09 01:04:47Z bmc $
003:          ---------------------------------------------------------------------------
004:          This software is released under a BSD-style license:
005:
006:          Copyright (c) 2004-2007 Brian M. Clapper. All rights reserved.
007:
008:          Redistribution and use in source and binary forms, with or without
009:          modification, are permitted provided that the following conditions are
010:          met:
011:
012:          1. Redistributions of source code must retain the above copyright notice,
013:             this list of conditions and the following disclaimer.
014:
015:          2. The end-user documentation included with the redistribution, if any,
016:             must include the following acknowlegement:
017:
018:                "This product includes software developed by Brian M. Clapper
019:                (bmc@clapper.org, http://www.clapper.org/bmc/). That software is
020:                copyright (c) 2004-2007 Brian M. Clapper."
021:
022:             Alternately, this acknowlegement may appear in the software itself,
023:             if wherever such third-party acknowlegements normally appear.
024:
025:          3. Neither the names "clapper.org", "curn", nor any of the names of the
026:             project contributors may be used to endorse or promote products
027:             derived from this software without prior written permission. For
028:             written permission, please contact bmc@clapper.org.
029:
030:          4. Products derived from this software may not be called "curn", nor may
031:             "clapper.org" appear in their names without prior written permission
032:             of Brian M. Clapper.
033:
034:          THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
035:          WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
036:          MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN
037:          NO EVENT SHALL BRIAN M. CLAPPER BE LIABLE FOR ANY DIRECT, INDIRECT,
038:          INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
039:          NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
040:          DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
041:          THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
042:          (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
043:          THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
044:        \*---------------------------------------------------------------------------*/
045:
046:        package org.clapper.curn.plugins;
047:
048:        import org.clapper.curn.Constants;
049:        import org.clapper.curn.CurnConfig;
050:        import org.clapper.curn.CurnException;
051:        import org.clapper.curn.FeedInfo;
052:        import org.clapper.curn.FeedConfigItemPlugIn;
053:        import org.clapper.curn.PostFeedParsePlugIn;
054:        import org.clapper.curn.parser.RSSChannel;
055:        import org.clapper.curn.parser.RSSItem;
056:
057:        import org.clapper.util.classutil.ClassUtil;
058:        import org.clapper.util.config.ConfigurationException;
059:        import org.clapper.util.html.HTMLUtil;
060:        import org.clapper.util.text.TextUtil;
061:        import org.clapper.util.logging.Logger;
062:
063:        import java.io.IOException;
064:        import java.io.StringReader;
065:
066:        import java.util.ArrayList;
067:        import java.util.Collection;
068:        import java.util.Map;
069:        import java.util.HashMap;
070:        import java.util.HashSet;
071:        import java.util.Iterator;
072:
073:        import java.util.regex.Matcher;
074:        import java.util.regex.Pattern;
075:        import java.util.regex.PatternSyntaxException;
076:        import org.clapper.curn.FeedCache;
077:        import org.clapper.util.misc.MultiValueMap;
078:
079:        /**
080:         * The <tt>ArticleFilterPlugIn</tt> provides per-feed filtering capabilities.
081:         * It can be used to filter out items (articles) that match one or more regular
082:         * expressions.
083:         *
084:         * <p>The filtering syntax is (shamelessly) adapted from the
085:         * <a href="http://offog.org/code/rawdog.html" target="_top"><i>rawdog</i></a>
086:         * RSS reader's
087:         * <a href="http://offog.org/files/rawdog-plugins/article-filter.py" target="_top"><i>article-filter</i></a>
088:         * plug-in. (<i>rawdog</i> is similar to <i>curn</i>: It's
089:         * a command line-driven RSS reader, written in Python.) A feed filter is
090:         * configured by adding an <tt>ArticleFilter</tt> property to the
091:         * feed's configuration section. The property's value consists of one or
092:         * more filter command sequences, separated by ";" characters. (The ";" must
093:         * be surrounded by white space; see below.) Each filter command sequence
094:         * is of this form:</p>
095:         *
096:         *
097:         * <blockquote>show|hide [<i>field</i> '<i>regexp</i>' [<i>field</i> '<i>regexp</i>' ...]]</blockquote>
098:         *
099:         * <p><i>field</i> can be one of:</p>
100:         *
101:         * <ul>
102:         *   <li> author: search the author field
103:         *   <li> title: search the title field
104:         *   <li> summary: search the summary, or description, field
105:         *   <li> text: search the full content, if available
106:         *   <li> category: search the article's category (or categories)
107:         *   <li> any: search all fields
108:         * </ul>
109:         *
110:         * <p>Each regular expression <i>must</i> be enclosed in single quotes.</p>
111:         *
112:         * <p>For example:</p>
113:         *
114:         * <blockquote><pre>
115:         * hide author 'Raymond Luxury-yacht' ; show author 'Arthur .Two-sheds. Jackson'
116:         * </pre></blockquote>
117:         *
118:         * <p>If the command is "hide", then the entry will be hidden if the
119:         * specified field matches the regular expression. If the command is
120:         * "show", then the entry will be shown if the field matches the regular
121:         * expression. If there are no fields or regular expressions, then the
122:         * command is a wildcard match. That is:</p>
123:         *
124:         * <blockquote><pre>hide</pre></blockquote>
125:         *
126:         * <p>is equivalent to:</p>
127:         *
128:         * <blockquote><pre>hide any '.*'</pre></blockquote>
129:         *
130:         * <p>and:</p>
131:         *
132:         * <blockquote><pre>show</pre></blockquote>
133:         *
134:         * <p>is equivalent to:</p>
135:         *
136:         * <blockquote><pre>show any '.*'</pre></blockquote>
137:         *
138:         * <p>Wildcard matches are useful in situations where you want to
139:         * hide or show "everything but ...". See the examples, below, for
140:         * details.</p>
141:         *
142:         * <p>All filtering commands are processed, and the end result is what defines
143:         * whether a given entry is suppressed or not. Regular expressions are
144:         * matched in a case-blind fashion. The match logic also
145:         *
146:         * <ul>
147:         *   <li>ignores any embedded newlines in article contents
148:         *   <li>(temporarily) strips all HTML from the article text before matching
149:         * </ul>
150:         *
151:         * <p>You can use multiple <tt>ArticleFilter</tt> parameters per feed (as long
152:         * as they have unique suffixes. All filters are applied to each article to
153:         * determine whether the article should be filtered out or not.</p>
154:         *
155:         * <h3>Examples</h3>
156:         *
157:         * <p>Some examples will help clarify the syntax.</p>
158:         *
159:         * <p>For example, the following set of commands hide all articles with the
160:         * phrase "mash-up" (because mash-ups bore me):</p>
161:         *
162:         * <blockquote>
163:         * <pre>
164:         * ArticleFilter: hide any 'mash[- \t]?up'
165:         * </pre>
166:         * </blockquote>
167:         *
168:         * <p>The following, more complicated, entry hides everything by author
169:         * "Joe Blow", unless the title has the word "rant" in it ('cause his rants
170:         * are hilarious):</p>
171:         *
172:         * <blockquote>
173:         * <pre>
174:         * ArticleFilter: hide author '^joe *blow$' ; show author '^joe *blow$' title rant
175:         * </pre>
176:         * </blockquote>
177:         *
178:         * <p>Finally, this example hides everything except articles by Moe Howard:</p>
179:         *
180:         * <blockquote>
181:         * <pre>
182:         * ArticleFilter: hide ; show author '^moe *howard$'
183:         * </pre>
184:         * </blockquote>
185:         *
186:         * @version <tt>$Revision: 7041 $</tt>
187:         */
188:        public class ArticleFilterPlugIn implements  FeedConfigItemPlugIn,
189:                PostFeedParsePlugIn {
190:            /*----------------------------------------------------------------------*\
191:                                     Private Constants
192:            \*----------------------------------------------------------------------*/
193:
194:            private static final String VAR_ITEM_FILTER = "ArticleFilter";
195:            private static final char COMMAND_DELIM = ';';
196:            private static final String STR_COMMAND_DELIM = "" + COMMAND_DELIM;
197:
198:            private static Map<String, Field> FIELD_NAME_MAP = // NOPMD
199:            new HashMap<String, Field>();
200:            static {
201:                for (Field field : Field.values())
202:                    FIELD_NAME_MAP.put(field.toString().toLowerCase(), field);
203:            }
204:
205:            private static Map<String, Command> COMMAND_MAP = // NOPMD
206:            new HashMap<String, Command>();
207:            static {
208:                for (Command command : Command.values())
209:                    COMMAND_MAP.put(command.toString().toLowerCase(), command);
210:            }
211:
212:            /*----------------------------------------------------------------------*\
213:                                      Private Classes
214:            \*----------------------------------------------------------------------*/
215:
216:            private static enum Command {
217:                HIDE, SHOW
218:            }
219:
220:            private static enum Field {
221:                AUTHOR, TITLE, SUMMARY, TEXT, CATEGORY, ANY
222:            }
223:
224:            private class FieldMatchRule {
225:                private Collection<Field> fields = new HashSet<Field>();
226:                private Pattern regex;
227:
228:                FieldMatchRule() {
229:                    // Nothing to do
230:                }
231:
232:                public FieldMatchRule addField(Field field) {
233:                    fields.add(field);
234:                    return this ;
235:                }
236:
237:                public Collection<Field> getFields() {
238:                    return fields;
239:                }
240:
241:                public Pattern getRegex() {
242:                    return regex;
243:                }
244:
245:                public void setRegex(Pattern regex) {
246:                    this .regex = regex;
247:                }
248:
249:                @Override
250:                public String toString() {
251:                    StringBuilder buf = new StringBuilder();
252:                    String sep = "<";
253:                    for (Field field : fields) {
254:                        buf.append(sep);
255:                        buf.append(field);
256:                        sep = ",";
257:                    }
258:
259:                    buf.append("> '");
260:                    buf.append(regex.toString());
261:                    buf.append("'");
262:
263:                    return buf.toString();
264:                }
265:            }
266:
267:            private class MatchRule {
268:                private Command command;
269:                private String unparsedFilter;
270:                private Collection<FieldMatchRule> matchRules = new ArrayList<FieldMatchRule>();
271:
272:                MatchRule(Command command, String unparsedFilter) {
273:                    this .command = command;
274:                    this .unparsedFilter = unparsedFilter;
275:                }
276:
277:                private void addFieldRule(FieldMatchRule rule) {
278:                    matchRules.add(rule);
279:                }
280:
281:                private Collection<FieldMatchRule> getFieldRules() {
282:                    return matchRules;
283:                }
284:
285:                private String getUnparsedFilter() {
286:                    return unparsedFilter;
287:                }
288:
289:                private Command getCommand() {
290:                    return command;
291:                }
292:
293:                @Override
294:                public String toString() {
295:                    StringBuilder buf = new StringBuilder();
296:
297:                    buf.append(command);
298:                    for (FieldMatchRule fieldRule : matchRules) {
299:                        buf.append(" ");
300:                        buf.append(fieldRule.toString());
301:                    }
302:
303:                    return buf.toString();
304:                }
305:            }
306:
307:            private class FeedFilterRuleset implements  Iterable<MatchRule> {
308:                private Collection<MatchRule> filterRules = new ArrayList<MatchRule>();
309:
310:                FeedFilterRuleset() {
311:                    // Nothing to do
312:                }
313:
314:                public void add(MatchRule rule) {
315:                    filterRules.add(rule);
316:                }
317:
318:                @Override
319:                public String toString() {
320:                    return filterRules.toString();
321:                }
322:
323:                public Iterator<MatchRule> iterator() {
324:                    return filterRules.iterator();
325:                }
326:            }
327:
328:            /*----------------------------------------------------------------------*\
329:                                    Private Data Items
330:            \*----------------------------------------------------------------------*/
331:
332:            /**
333:             * For log messages
334:             */
335:            private static final Logger log = new Logger(
336:                    ArticleFilterPlugIn.class);
337:
338:            /**
339:             * Per-feed match rules
340:             */
341:            private MultiValueMap<FeedInfo, FeedFilterRuleset> perFeedMatchRules = new MultiValueMap<FeedInfo, FeedFilterRuleset>();
342:
343:            /*----------------------------------------------------------------------*\
344:                                        Constructor
345:            \*----------------------------------------------------------------------*/
346:
347:            /**
348:             * Default constructor (required).
349:             */
350:            public ArticleFilterPlugIn() {
351:                // Nothing to do
352:            }
353:
354:            /*----------------------------------------------------------------------*\
355:                       Public Methods Required by *PlugIn Interfaces
356:            \*----------------------------------------------------------------------*/
357:
358:            /**
359:             * Get a displayable name for the plug-in.
360:             *
361:             * @return the name
362:             */
363:            public String getPlugInName() {
364:                return "Article Filter";
365:            }
366:
367:            /**
368:             * Get the sort key for this plug-in.
369:             *
370:             * @return the sort key string.
371:             */
372:            public String getPlugInSortKey() {
373:                return ClassUtil.getShortClassName(getClass().getName());
374:            }
375:
376:            /**
377:             * Initialize the plug-in. This method is called before any of the
378:             * plug-in methods are called.
379:             *
380:             * @throws CurnException on error
381:             */
382:            public void initPlugIn() throws CurnException {
383:            }
384:
385:            /**
386:             * Called immediately after <i>curn</i> has read and processed a
387:             * configuration item in a "feed" configuration section. All
388:             * configuration items are passed, one by one, to each loaded plug-in.
389:             * If a plug-in class is not interested in a particular configuration
390:             * item, this method should simply return without doing anything. Note
391:             * that some configuration items may simply be variable assignment;
392:             * there's no real way to distinguish a variable assignment from a
393:             * blessed configuration item.
394:             *
395:             * @param sectionName  the name of the configuration section where
396:             *                     the item was found
397:             * @param paramName    the name of the parameter
398:             * @param config       the active configuration
399:             * @param feedInfo     partially complete <tt>FeedInfo</tt> object
400:             *                     for the feed. The URL is guaranteed to be
401:             *                     present, but no other fields are.
402:             *
403:             * @return <tt>true</tt> to continue processing the feed,
404:             *         <tt>false</tt> to skip it
405:             *
406:             * @throws CurnException on error
407:             *
408:             * @see CurnConfig
409:             * @see FeedInfo
410:             * @see FeedInfo#getURL
411:             */
412:            public boolean runFeedConfigItemPlugIn(String sectionName,
413:                    String paramName, CurnConfig config, FeedInfo feedInfo)
414:                    throws CurnException {
415:                try {
416:                    if (paramName.startsWith(VAR_ITEM_FILTER)) {
417:                        String rawValue = config.getRawValue(sectionName,
418:                                paramName);
419:                        perFeedMatchRules.put(feedInfo, parseFilterSpec(
420:                                sectionName, paramName, rawValue));
421:                    }
422:
423:                    return true;
424:                }
425:
426:                catch (ConfigurationException ex) {
427:                    throw new CurnException(ex);
428:                }
429:            }
430:
431:            /**
432:             * Called immediately after a feed is parsed, but before it is
433:             * otherwise processed. This method can return <tt>false</tt> to signal
434:             * <i>curn</i> that the feed should be skipped. For instance, a plug-in
435:             * that filters on the parsed feed data could use this method to weed
436:             * out non-matching feeds before they are downloaded. Similarly, a
437:             * plug-in that edits the parsed data (removing or editing individual
438:             * items, for instance) could use method to do so.
439:             *
440:             * @param feedInfo  the {@link FeedInfo} object for the feed that
441:             *                  has been downloaded and parsed.
442:             * @param feedCache the feed cache
443:             * @param channel   the parsed channel data
444:             *
445:             * @return <tt>true</tt> if <i>curn</i> should continue to process the
446:             *         feed, <tt>false</tt> to skip the feed. A return value of
447:             *         <tt>false</tt> aborts all further processing on the feed.
448:             *         In particular, <i>curn</i> will not pass the feed along to
449:             *         other plug-ins that have yet to be notified of this event.
450:             *
451:             * @throws CurnException on error
452:             *
453:             * @see RSSChannel
454:             * @see FeedInfo
455:             */
456:            public boolean runPostFeedParsePlugIn(FeedInfo feedInfo,
457:                    FeedCache feedCache, RSSChannel channel)
458:                    throws CurnException {
459:                Collection<FeedFilterRuleset> rules = perFeedMatchRules
460:                        .getCollection(feedInfo);
461:
462:                if ((rules != null) && (rules.size() > 0)) {
463:                    for (RSSItem item : channel.getItems()) {
464:                        if (nukeItem(item, rules, feedInfo)) {
465:                            // Since getItems() returns a copy of the list of
466:                            // items, this call will not cause a
467:                            // ConcurrentModificationException to be thrown.
468:
469:                            log.debug("Feed \"" + feedInfo.getURL()
470:                                    + "\": Filtering out item \""
471:                                    + item.getTitle() + "\"");
472:                            channel.removeItem(item);
473:                        }
474:                    }
475:                }
476:
477:                return true;
478:            }
479:
480:            /*----------------------------------------------------------------------*\
481:                                      Private Methods
482:            \*----------------------------------------------------------------------*/
483:
484:            private FeedFilterRuleset parseFilterSpec(String sectionName,
485:                    String paramName, String rawValue) throws CurnException {
486:                FeedFilterRuleset result = new FeedFilterRuleset();
487:                String[] tokens = parseFilterTokens(rawValue);
488:
489:                try {
490:                    int i = 0;
491:                    while (i < tokens.length) {
492:                        // Strip off the first token, which should be the command.
493:
494:                        String sCommand = tokens[i++];
495:                        Command cmd = COMMAND_MAP.get(sCommand);
496:                        if (cmd == null) {
497:                            throw new CurnException(
498:                                    Constants.BUNDLE_NAME,
499:                                    "ArticleFilterPlugIn.badFilterCommand",
500:                                    "Configuration section \"{0}\": Value for "
501:                                            + "parameter \"{1}\" has a bad command field of "
502:                                            + "\"{2}\".", new Object[] {
503:                                            sectionName, paramName, sCommand });
504:                        }
505:
506:                        MatchRule matchRule = new MatchRule(cmd, rawValue);
507:
508:                        // Check for a wildcard field.
509:
510:                        if (tokens[i].equals(STR_COMMAND_DELIM)) {
511:                            // Wild card rule.
512:
513:                            FieldMatchRule fieldRule = new FieldMatchRule();
514:                            fieldRule.addField(Field.AUTHOR).addField(
515:                                    Field.TITLE).addField(Field.SUMMARY)
516:                                    .addField(Field.TEXT).addField(
517:                                            Field.CATEGORY).setRegex(
518:                                            compileRegex(".*"));
519:                            i++;
520:                            matchRule.addFieldRule(fieldRule);
521:                        }
522:
523:                        else {
524:                            // No wildcard. Strip off the fields and regular
525:                            // expressions, until we hit the next ";" or an end of
526:                            // line.
527:
528:                            while ((i < tokens.length)
529:                                    && (!tokens[i].equals(STR_COMMAND_DELIM))) {
530:                                // Strip off field name.
531:
532:                                String strField = tokens[i++];
533:                                String uncompiledRegex = tokens[i++];
534:                                FieldMatchRule fieldRule = new FieldMatchRule();
535:                                Field field = FIELD_NAME_MAP.get(strField);
536:                                if (field == null) {
537:                                    throw new CurnException(
538:                                            Constants.BUNDLE_NAME,
539:                                            "ArticleFilterPlugIn.badRSSField",
540:                                            "Configuration section \"{0}\": Value for "
541:                                                    + "parameter \"{1}\" has a bad RSS field of "
542:                                                    + "\"{2}\".", new Object[] {
543:                                                    sectionName, paramName,
544:                                                    strField });
545:                                }
546:
547:                                if (field == Field.ANY) {
548:                                    fieldRule.addField(Field.AUTHOR).addField(
549:                                            Field.TITLE)
550:                                            .addField(Field.SUMMARY).addField(
551:                                                    Field.TEXT).addField(
552:                                                    Field.CATEGORY);
553:                                }
554:
555:                                else {
556:                                    fieldRule.addField(field);
557:                                }
558:
559:                                fieldRule
560:                                        .setRegex(compileRegex(uncompiledRegex));
561:                                matchRule.addFieldRule(fieldRule);
562:                            }
563:
564:                            i++;
565:                        }
566:
567:                        result.add(matchRule);
568:                    }
569:                }
570:
571:                catch (ArrayIndexOutOfBoundsException ex) {
572:                    throw new CurnException(Constants.BUNDLE_NAME,
573:                            "ArticleFilterPlugIn.wrongNumberOfFields",
574:                            "Configuration section \"{0}\": Value for parameter "
575:                                    + "\"{1}\" is missing at least one field.",
576:                            new Object[] { sectionName, paramName });
577:                }
578:
579:                return result;
580:            }
581:
582:            private boolean nukeItem(RSSItem item,
583:                    Collection<FeedFilterRuleset> rulesets, FeedInfo feedInfo) {
584:                boolean match = false;
585:
586:                for (FeedFilterRuleset ruleset : rulesets) {
587:                    match = nukeItem(item, ruleset, feedInfo);
588:                    if (match)
589:                        break;
590:                }
591:
592:                return match;
593:            }
594:
595:            private boolean nukeItem(RSSItem item, FeedFilterRuleset ruleset,
596:                    FeedInfo feedInfo) {
597:                boolean killItem = false;
598:                String itemId = feedInfo.getURL() + ", " + item.getTitle();
599:
600:                log.debug("item " + itemId + ": checking filter: "
601:                        + ruleset.toString());
602:                for (MatchRule rule : ruleset) {
603:                    boolean match = true;
604:                    boolean hide = (rule.getCommand() == Command.HIDE);
605:
606:                    for (FieldMatchRule fieldRule : rule.getFieldRules()) {
607:                        log.debug("item=" + item.getTitle() + ", command="
608:                                + rule.getCommand() + ", "
609:                                + fieldRule.toString());
610:
611:                        StringBuilder buf = new StringBuilder();
612:
613:                        for (Field field : fieldRule.getFields()) {
614:                            switch (field) {
615:                            case AUTHOR:
616:                                Collection<String> authors = item.getAuthors();
617:                                if ((authors != null) && (authors.size() > 0))
618:                                    buf.append(TextUtil.join(authors, " "));
619:                                break;
620:
621:                            case CATEGORY:
622:                                Collection<String> cats = item.getCategories();
623:                                if ((cats != null) && (cats.size() > 0))
624:                                    buf.append(TextUtil.join(cats, " "));
625:                                break;
626:
627:                            case TITLE:
628:                                buf.append(item.getTitle());
629:                                break;
630:
631:                            case SUMMARY:
632:                                buf.append(item.getSummary());
633:                                break;
634:
635:                            case TEXT:
636:                                buf.append(item.getFirstContentOfType(
637:                                        "text/plain", "text/html"));
638:                                break;
639:
640:                            default:
641:                                assert (false);
642:                                break;
643:                            }
644:                        }
645:
646:                        String toMatch = HTMLUtil.textFromHTML(buf.toString());
647:                        Pattern regex = fieldRule.getRegex();
648:                        Matcher matcher = regex.matcher(toMatch);
649:                        boolean regexMatches = matcher.find();
650:                        log.debug("regex '"
651:                                + regex.toString()
652:                                + "' "
653:                                + (regexMatches ? "matches: "
654:                                        : "doesn't match: ") + toMatch);
655:                        if (!regexMatches)
656:                            match = false;
657:                    }
658:
659:                    if (match)
660:                        killItem = hide;
661:                }
662:
663:                log.debug("item: " + itemId + ", kill=" + killItem);
664:                return killItem;
665:            }
666:
667:            private String[] parseFilterTokens(String rawValue)
668:                    throws CurnException {
669:                ArrayList<String> tokens = new ArrayList<String>();
670:
671:                StringReader r = new StringReader(rawValue);
672:                StringBuilder buf = new StringBuilder();
673:                int col = 0;
674:                try {
675:                    int ich;
676:                    while ((ich = r.read()) != -1) {
677:                        col++;
678:
679:                        char ch = (char) ich;
680:                        if (ch == '\'') {
681:                            // Look for ending quote.
682:
683:                            buf.setLength(0);
684:                            while (((ich = r.read()) != -1)
685:                                    && (((char) ich) != '\'')) {
686:                                col++;
687:                                buf.append((char) ich);
688:                            }
689:
690:                            if (ich == -1) {
691:                                throw new CurnException(
692:                                        Constants.BUNDLE_NAME,
693:                                        "ArticleFilterPlugIn.unmatchedQuote",
694:                                        "Unmatched single quote at column {0} in \"{1}\"",
695:                                        new Object[] { col, rawValue });
696:                            }
697:
698:                            tokens.add(buf.toString());
699:                        }
700:
701:                        else if (ch == COMMAND_DELIM) {
702:                            tokens.add(String.valueOf(ch));
703:                        }
704:
705:                        else if (Character.isWhitespace(ch)) {
706:                            // Skip
707:                        }
708:
709:                        else {
710:                            // Keep going until we hit white space or the end of
711:                            // the line.
712:
713:                            buf.setLength(0);
714:                            buf.append(ch);
715:                            while (((ich = r.read()) != -1)
716:                                    && (((char) ich) != '\'')
717:                                    && (!Character.isWhitespace((char) ich))) {
718:                                col++;
719:                                buf.append((char) ich);
720:                            }
721:
722:                            tokens.add(buf.toString());
723:                        }
724:                    }
725:                }
726:
727:                catch (IOException ex) {
728:                    // Shouldn't happen
729:
730:                    throw new CurnException(
731:                            "Huh? IOException reading StringReader.", ex);
732:                }
733:
734:                return tokens.toArray(new String[tokens.size()]);
735:            }
736:
737:            private Pattern compileRegex(String strRegex) throws CurnException {
738:                try {
739:                    return Pattern.compile(strRegex, Pattern.CASE_INSENSITIVE);
740:                }
741:
742:                catch (PatternSyntaxException ex) {
743:                    throw new CurnException(Constants.BUNDLE_NAME,
744:                            "ArticleFilterPlugIn.badRegex",
745:                            "\"{0}\" is an invalid regular expression",
746:                            new Object[] { strRegex }, ex);
747:                }
748:            }
749:        }
www.java2java.com | Contact Us
Copyright 2009 - 12 Demo Source and Support. All rights reserved.
All other trademarks are property of their respective owners.