| java.lang.Object
 com.sun.portal.providers.urlscraper.Fetcher
| Fetcher | | public class Fetcher (Code) | | This class fetches and stores the content from an arbitrary url
|
| Inner Class :static class domainPathEntry | |
| Method Summary | |
| StringBuffer | fetch(FetcherThread ft, String u)
Fetch the contents from the specified URL. | | public static String | getContentEncoding(String contentType, byte[] bytes, String MIMEType, String profileCharset)
Gets the charset
This method determines the charset based on the
contentType header if it is available (only applies to http(s) urls),
or from the inputEncoding property if it is non-blank, or from the meta
tag in content, e.g. | | static String | getContentEncodingFromContentBytes(byte[] contentBytes)
| | public StringBuffer | getFilteredContent(int timeOut, HttpServletRequest req, HttpServletResponse res, String urlAsString)
| | public void | logout(int timeOut, HttpServletRequest req, HttpServletResponse res)
| | static byte[] | readContent(InputStream in, int contentLength)
|
| KEY_COOKIES_TO_FORWARD_ALL | | final public static String KEY_COOKIES_TO_FORWARD_ALL(Code) | | wether to Forward all Cookies as Boolean
|
| KEY_COOKIES_TO_FORWARD_LIST | | final public static String KEY_COOKIES_TO_FORWARD_LIST(Code) | | if only selcted Cookies will be forwarded, the list of cookies to be forwarded as List
|
| KEY_FORM_DATA | | final public static String KEY_FORM_DATA(Code) | | The formData as String e.g: login=[uid]&passwd=[password] wher the characters in square bracket are properties available in the channel.
|
| KEY_HTTP_AUTH | | final public static String KEY_HTTP_AUTH(Code) | | isHttpAuth , set true only if HttpAuth
|
| KEY_HTTP_AUTH_PASSWORD | | final public static String KEY_HTTP_AUTH_PASSWORD(Code) | | the password for Http Auth
|
| KEY_HTTP_AUTH_UID | | final public static String KEY_HTTP_AUTH_UID(Code) | | The Uid for Http Auth
|
| KEY_INPUT_ENCODING | | final public static String KEY_INPUT_ENCODING(Code) | | The Input encoding as String
|
| KEY_LOGIN_FORM_DATA | | final public static String KEY_LOGIN_FORM_DATA(Code) | | The form data to be sent to during login if provided e.g: login=[uid]&passwd=[password]
|
| KEY_LOGIN_URL | | final public static String KEY_LOGIN_URL(Code) | | login URL as string . If this is provided it will be used once to authenticate for the first time
|
| KEY_LOGOUT_URL | | final public static String KEY_LOGOUT_URL(Code) | | logoutUrl as string . If this is provided it will be used logout
|
| KEY_RESOURCE_BUNDLE | | final public static String KEY_RESOURCE_BUNDLE(Code) | | As ResourceBundle
|
| KEY_RULESET_ID | | final public static String KEY_RULESET_ID(Code) | | The RulesetId as String
|
| loop | | int loop(Code) | | just to store the conetnType, required for ContentFilter
|
| getContentEncoding | | public static String getContentEncoding(String contentType, byte[] bytes, String MIMEType, String profileCharset)(Code) | | Gets the charset
This method determines the charset based on the
contentType header if it is available (only applies to http(s) urls),
or from the inputEncoding property if it is non-blank, or from the meta
tag in content, e.g. meta tag in html, xml or wml header if they are
available (only applies to HTML, XML, WML).
String charsetor null if charset cannot be determined
Parameters:
profileCharset -
Parameters:
contentType - If http(s) urls, null otherwise
Parameters:
bytes - Bytes from the scraped content
Parameters:
MIMEType - MIMEType for the content |
| getContentEncodingFromContentBytes | | static String getContentEncodingFromContentBytes(byte[] contentBytes)(Code) | | Gets the charset from content
This method determines the charset based on
meta tag in content
Parameters:
contentBytes - Bytes from the scraped content String charset or null if charset cannot be determined |
|
|