v0.2
v0.1
Preparing search index...
The search index is not available
LangChain.js - v0.2.12
LangChain.js
langchain-textsplitters
TokenTextSplitterParams
Interface TokenTextSplitterParams
interface
TokenTextSplitterParams
{
allowedSpecial
:
string
[]
|
"all"
;
chunkOverlap
:
number
;
chunkSize
:
number
;
disallowedSpecial
:
string
[]
|
"all"
;
encodingName
:
TiktokenEncoding
;
keepSeparator
:
boolean
;
lengthFunction
?:
(
(
text
:
string
)
=>
number
)
|
(
(
text
:
string
)
=>
Promise
<
number
>
)
;
}
Hierarchy (
view full
)
TextSplitterParams
TokenTextSplitterParams
Implemented by
TokenTextSplitter
Index
Properties
allowed
Special
chunk
Overlap
chunk
Size
disallowed
Special
encoding
Name
keep
Separator
length
Function?
Properties
allowed
Special
allowed
Special
:
string
[]
|
"all"
chunk
Overlap
chunk
Overlap
:
number
chunk
Size
chunk
Size
:
number
disallowed
Special
disallowed
Special
:
string
[]
|
"all"
encoding
Name
encoding
Name
:
TiktokenEncoding
keep
Separator
keep
Separator
:
boolean
Optional
length
Function
length
Function
?:
(
(
text
:
string
)
=>
number
)
|
(
(
text
:
string
)
=>
Promise
<
number
>
)
Settings
Member Visibility
Protected
Inherited
Theme
OS
Light
Dark
On This Page
Properties
allowed
Special
chunk
Overlap
chunk
Size
disallowed
Special
encoding
Name
keep
Separator
length
Function
LangChain.js - v0.2.12
Loading...
"
"